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ABSTRACT 

This report describes initial steps taken in a 
research project aimed at gaining insight into the nature of 
effective foreign language teaching. Two systems for observing and 
describing classroom behavior of teachers and students were developed 
in this experiment. These systems were used to observe 
university- level French classes. Data collected from the observations 
are analyzed to produce profiles of typical classroom behavior and 
also to attempt to differentiate between behaviors of more effective 
and less effective teachers. The authors comment on the potential 
application of the systems in teacher training programs. More than 20 
statistical tables are included and serve as the basis for concluding 
remarks. A bibliography is provided. (RL) 



ERjt 



January 31, IS 71 



FINAL REPORT 



Sor. <^*2 



- 3 - 



oo 

oo 

- 4 * 

o 

ca 

Uli 



Contract No. 0EC-0-9-097740-3743(014) 

II.S. DEPARTMENT OF HEALTH, EDUCATION 4 WELFARE 
OFFICE OF EDUCATION 



THIS DOCUMENT HAS BEEN REPRODUCED EXACTLY AS RECEIVED FROM THE 
PERSON OR ORGANIZATION ORIGINATING «. POINTS OF VIEW OR OPINIONS 
STATED DO NOT NECESSARILY REPRESENT OFFICIAL OFFICE OF EDUCATION 
POSITION OR P0UC v . 



DEVELOPMENT OF COMPUTER-ASSISTED OBSERVATIONAL 
SYSTEMS FOR TEACHER TRAINING 

Guy C. Capelle, Robert J. Jarvella, and Eleanor Revelle 



o 




(< 

o 

o 



J 

I ® 

'eric 



Center for Research on Language and Language Behavior 
The University of Michigan 
220 E. Huron 

Ann Arbor, Michigan 48108 



Office of Education 
Institute of International Studies 
U.S' Department of Health, Education, and Welfare 



1 77 ? a 



0 



I 



DEVELOPMENT OF COMPUTER-ASSISTED OBSERVATIONAL SYSTEMS 

FOR TEACHER TRAINING 1 
2 

Guy C. Capelle, Robert J. Jarvella, and Eleanor Revelle 

Center for Research on Language and Language Behavior 
The University of Michigan 



Two systems for observing classroom behavior of 
teachers and students were developed. These systems 
were used to observe university-level classes in which 
French was being taught to English-speaking under- 
graduates. Data collected from the observations 
were analyzed to produce profiles of typical classroom 
behavior and also to attempt to differentiate between 
behaviors of more effective and less effective teachers. 
The systems appear to have considerable potential for 
teacher-training . 



Introduction 

General Objectives of the Research 

The present research is to be considered as an initial step in a 
long-term project aimed at gaining insight into the nature of effective 
foreign language teaching. Investigators have attempted to describe and 
quantify teachers 1 classroom behavior in order to correlate it with some 
measure of effectiveness , usually as reflected in student achievement. 

This is the approach which has been followed here. If it proves adequate, 
it should eventually be possible a) to determine from the observational 
data what constitutes effective teaching, b) to tailor teacher training 
to the needs of individual trainees, and 3) to assess the result of 
training by comparing the trainees 1 behavior before and after training. 

In accordance with these long-range goals, the first aim of the 
present research was to develop, test, and refine instruments directed 
specifically at describing verbal behavior in the language classroom 
situation. It was decided to limit these instruments initially to 
handling the particular problems of the beginning level of language 
teaching . 

Need for This Research 



Although Rosenshine (1970) reports some 51 studies based on the observa- 
tion of classroom bheavior, which he considers the foremost in recent years, 
not a single study has to do with the foreign language classroom. Apart 
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from two adaptions of the Flanders system (Moskowitz, 1966, and Jar- 
vitts, the investigators are not aware of any observational system which 
could describe an essential aspect of language teaching, the use of the 
foreign language in the classroom. As Biddle remarks in the conclusion 
of his review of methods and concepts in classroom research (1967) 

"...the major field as yet untapped in classroom research is that of 
linguistics. Linguistic usage differentiates three domains of concepts: 
semantics (the study of meaning) , phonology (the study of sounds) , and 
syntax (the study of language forms and sequences) . From a linguistic 
point of view, the majority of the codes suggested in the studies reviewed 
are semantic in orientation." 

It appeared to the present investigators that the study of language 
use in the classroom (amount, type, correctness, sequence) was an in- 
dispensable addition t o any one of the existing approaches if one 
desired insight into the teaching of a subject which is at the same 
time the medium through which the teaching is done. It was also noted 
that none of the existing systems seemed adequate to describe some of 
the techniques most widely used in language teaching (repetition, 
choral responses, teaching the foreign language or teaching about the 
foreign language, use of the student’s native language, drills). 

This research thus appears as an application to the foreign language 
classroom of some of the studies on classroom climate as well as an effort 
to analyze some of the problems specific to foreign language teaching. 

Basic Assumptions 

The belief that new insight into the nature of effective language 
teaching can best be gained by making detailed behavioral records of 
what teaching is is based on the following assumptions: 

— that one of the best ways to learn what teaching is is to study 
classroom behavior, 

—that this behavior can be observed, i.e., that it can be 
categorized into discrete units and quantified. 

— that the observable teacher behavior has an effect on student 
learning. 

— that this effect is reflected in independent, obtainable 
measures of teacher effectiveness. 

Overview of Literature 



Investigators have attempted to clarify teacher-student interaction 
in the classroom in two main areas; cognitive processes (Smith & Meux, 
1962; Gallagher, 1965; Be Hack, Hyman, Kliegard & Smith, 1965) and social 
emotional climate (mostly Flanders). 
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Smith and Muux (1962) used a logical model to analyze teaching, 
Gallagher (1965) used an observational system in which a component was 
a thinking operation modeled on Guilford T s paradigm for the structure of 
the intellect. Bellack, et al. (1965) used Wittgenstein’s model of lan- 
guage games to analyze the teaching process into cyclical patterns of 
pedagogical moves. 

All three types of research focussed on the interactive sequence 
of moves and turned away from statis modes. But, whereas Smith and 
Meux and Gallagher tried to anticipate the sequences, Bellack, et al. 
attempted to construct them from the observed sequence of categories. 

The defining of larger units in terms of smaller units seems the best 
way to preserve reliability and this approach will be followed by the 
present investigators when they attempt to determine the significant 
patterns of language teaching. 

From H. H, Anderson to Ned A. Flanders, a generation of investigators 
has based its model on social psychology, and studied classroom social- 
emotional climate, those "generalized attitudes toward the teacher and 
the class that the pupils have in common in spite of individual differences" 
attitudes whose development is "an outgrowth of classroom social inter- 
action" (Flanders, 1966). 

Flanders presents the most comprehensive program of classroom ob- 
servation using a system of ten analytic, molecular categories reflecting 
immediately observable teacher and student activities whose frequencies 
are plotted on Markovian matrices to preserve their sequential properties. 
His analysis of teacher-student interaction in the classroom leads 
mostly to the testing of the effects of "direct" or "indirect" teacher 
influence on students — "indirect" — when the goals and the type of teach- 
ing activity change. 

Part of the present research draws heavily upon Flanders’ work. It 
also greatly benefited from research conducted at the Center for Research 
on Language and Language Behavior, University of Michigan, by Melvyn I. 
Semmel, et al. (1968), who developed a computer-assisted teacher training 
system (CATTS) , Their techniques for training observers and checking 
reliability, based on the use of a specially developed computer program 
(CONCODE) , provided imaginative and time-saving devices. 

Hypotheses of Teacher Effectiveness in this Study 

In this preliminary study the hypotheses tested concerned primarily 
the validity and predictive power of the two observational systems 
developed to describe a) teacher influence, b) teacher language teaching 
techniques, and c) the types of language used in the classroom. 

a. Teacher influence : The general hypothesis is based on Flanders’ 

concepts for describing teacher influence (Flanders, 1965). It was 
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hypothesized that encouraging students to speak , exerting indirect 
influence and showing concern for student participation and flexibil- 
ity in teaching would maximize effectiveness. 

b. Teacher techniques : The general hypothesis here is that tech- 

niques stimulating active language use and techniques confronting 
students with problems to solve in the foreign language would con- 
tribute to greater effectiveness. This implies that individual answers 

to questions are more instructive than mere repetition, or choral practice, 
that reciting is motfs productive than lecturing; that the use of "wh" 
questions results in more meaningful learning than the use of mere 
v, yes/no" questions. 

c. Amount, nature and type of language use . It was hypothesized 
that for the best learning to occur, the use of the foreign language 
should be maximal; that spontaneous language use by the students should 
be encouraged; and finally that the correctness of use is an important 
factor of effectiveness. 

Three independent measures of teacher effectiveness were to be 
correlated with a measure of predicted effectiveness based on each 
hypothesis: 

a. Classroom achievement measured by examination soon after the 
taking of observations; 

b. student ratings of teachers to be obtained at the end of 
formal instruction; 

c. observers’ ratings based on a detailed evaluation of teacher 
behavior. 

Of these three measures, student achievement was considered the 
most acceptable criterion. If the two observational systems were found 
reliable, could discriminate between teachers and serve as predictors 
of effectiveness, they could be used as valid instruments in the further 
stages of the long-term research project. 

Description of the Category Systems 

An observational system was seen as requiring categories which are 
unambiguously defined, mutually exclusive, and adequately representative 
of the behavior to be described. It was also intended that such a system 
be suitable for coding behavior in real time. An additional, self-imposed 
constraint was that it be a low-inference system, that is, that it employ 
analytic, specific and readily identifiable categories, thus minimizing 
observers’ subjective judgments and permitting comparable use of the 
system by other research teams. Detailed criteria were to be given to 
observers to facilitate category identification and maximize reliability. 
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Three observational systems had originally been planned for use: 
a) Flanders' ten-category system of interaction analysis to measure 
verbal behavior, as reflected in interaction ; b) a system specially 
designed to record verbal behavior as an indication of teaching techniques ; 
c) a third system designed to reflect the form and quality of the verbal 
behavior under investigation. Because of the overlap between Flanders' 
system and that aimed at reflecting teaching techniques, these first two 
proposed systems were collapsed into one (0S1) . This new system was 
designed to capture esssential aspects of the foreign language classroom, 
while preserving most of the potential offered by Flanders 1 system. 

Flanders' first and third categories, 1) Teacher accepts feelings, and 
3) Teacher accepts or uses ideas of students, were so infrequent that the 
were collapsed: the possibilities of expression would still be so 
limited at this stage of study that students would rarely word original 
thoughts in the foreign language, and teachers expand them. In addition, 
short praise tends to be used so frequently that it often becomes automatic 
and loses some of its reward value. Extended praise, on the other hand, 
could not be captured by a sequence of short praise tallies as it rarely 
lasts more than 3 sec. because of limitations in grammar and vocabulary 
at the beginning level. A cue in a drill situation must be kept distinct 
from an order, likewise, shaping of student response, another typical 
language teaching activity, needs to be categorized as separate from 
lecturing. These and other differences with Flanders' categories will 
become apparent in the following description of the categories prepared 
for the observers. 

Description of the First Observational System (OS) 

As in Flanders' model, the 0S1 system makes use of 10 macro-cate- 
gories: seven to indicate teacher behavior; two for student behavior; and 
a tenth for silence or confusion. The seven categories describing teacher 
behavior reflect to a considerable degree the "indirect-direct influence" 
distinction drawn by Flanders: the first four generally encourage student 
participation while the next three are more indicative of teacher influence 
and control. 

Seven of the ten macro-categories, however, are further subdivided 
to more adequately reflect foreign language teaching interaction and tech- 
niques. The macro-categories facilitate the coder's task by grouping 
together categories of the same type for a first decision. Then, sharper 
distinctions are made within categories for a second decision. Thus, a 
question by the teacher is categorized as 4, and is then more finely 
differentiated as either a conversation question, 4. , or a question about 
the target language, 4 2 « 

Category 1 is used to reflect positive and encouraging teacher 
comments . Its occurrence was fairly infrequent in the observational data 
of this study. More finely differentiated, sub-category 1^ indicates a) a 
teacher's acceptance of student feelings, b) a teacher's use of student 
ideas, and c) a teacher's tension releasing jokes or comments. Sub-category 
I 2 indicates a teacher's extensive praise of student responses, or 
encouragement to respond: 
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Examples of category 1^.* 

"Je sals que c'est difficile..." 

"Vous comprenez tout ce que je dis aujourd'hui." 

"Je suis content que vous aimiez tant ce dialogue." 

"Est-ce que vous comprenez?" (directed at the class in general. 

and showing real interest, i.e., not rhetorical) 

"Pardon." 

"Excusez-moi. " 

Examples of category 1^: 

"C'est une tres bonne question." 

"Oui, Monsieur." (Calling on student volunteering) 

"Qui peut repondre?" 

"Avez-vous des questions a me poser?" 

"Pardon?" 

"Oui." 

"Vous ne comprenez pas la question?": (directed at one student as 

encouragement to ask about what he does not understand) 

Category 2 is used to describe teacher response to student answers. 

It was employed quite frequently in the coding of this study's data. This 
category is also broken down into two parts: 2. indicates short words of 
praise, and 2^ denotes teacher reinforcement or student response by 
repetition or indicates a teacher's final move. The most frequent 
occurrence of category 2^ was in drill situations although it also occurred 
after students' response to teachers' questions. 

Examples of category 2^: 

"Oui, c'est qa." 

"Tres bien." 

Examples of category : 

Teacher repeats student's correct response as a final mode. 

Teacher repeats correct response for repetition. 

Teacher gives correct response without intent to correct a 
preceding error. 

Category 3 describes teacher model- giving behavior, generally within 
a drill situation. The subcategory 3^ denotes a teacher's model or example. 
Examples embedded within a lecture sequence are recorded as part of the 
lecture (5^) rather than separately as examples. Subcategory indicates 
a cue for student response in a drill sequence. 

Examples of category 3^: 

Teacher gives model for a drill. 

Teacher gives a model for repetition. 

Teacher reads from text. 
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Examples of category 

Teacher gives cue word in substitution drill. 

Teacher gives sentence to be transformed. 

Teacher gives sentence to be completed. 

Teacher asks question whose answer is highly restricted in form. 

Category 4 is used to indicate teacher probes . These questions may 

be of a conversational nature (4-) or may refer to the target language 
// \ J- 



Examples of category 4^: 

"Quelle heure est-il?" 

"Quel cours suivez-vous a une heure?" 

"Qu'est-ce que c'est qu'une librairie?" 

Examples of category 4 2 : 

"Quel est 1* inf ini t if de ce verbe?" 

"Que veut dire le mot 'delicat? 1 " 

"Comment traduisez-vous cette phrase?" 

Category 5 is undifferentiated and is used to indicate teacher 
lecturing. This includes giving facts or opinions and asking rhetorical 
questions. 

Examples of this category include: 

Teacher lectures about language, culture, etc. 

Teacher answers student initiated questions (9-, 9 2 ). 

Teacher answers his own questions. 

Teacher rewords student response where student response is 
the expected one rather than a novel idea. 

"Vous comprenez?" (no answer expected from students; automatic and 
showing no real concern) * 

Category 6 is also undifferentiated and is used to describe teacher 
directions. This also includes indications about general classroom 
procedures. 

Examples of this category include: 

"Ouvrez vos livres a la page..." 

"Traduisez cette phrase en anglais." 

"Repetez." "Encore." "Tout le monde." 

"Employez le conditionnel." 

"Nous allons etudier le verbe 'suivre. 1 " 

"Demain nous f inirons ces exercices." 
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Category 7 Is used to describe teacher corrections. The subscategory 
7^ indicates the shaping of behavior whether through correction of student 
responses or through giving clues. The subcategory 7« is used to describe 
teacher rejection of student response or behavior. It would also be used 
to indicate negative jokes made at students 1 expense. 

Examples of category 7^: 

Teacher corrects student in process of responding. 

Teacher corrects pronunciation with proper mode. 

Teacher provides first words of response as a clue. 

Teacher gives clues to aid student in correcting himself. 

Examples of category 7^: 

"Non, ce n'est pas 5 a." 

"Est-ce que c'est le conditionnel?" (as response to student* s 

incorrect answer) 

"II faut travailler un petit peu." 

Category 8 is used to indicate student responses . It is broken down 
into three parts: 8 ^ indicates individual repetition; 82 indicates choral 
repetition or answers; and 8 ^ indicates individual answers. 

Examples of category 8 ^: 

Student repeats drill model or correct response. 

Student uads from text. 

Student repeats after teacher reads. 

Examples of category 82 : 

Choral repetition of drill model or correct response. 

Choral reading cr dialogue repetition. 

Choral response in a drill situation. 

Examples of dategory 8 ^: 

Student responds in a drill situation. 

Student translates. 

Student answers questions (4^ or ^ 2 ^ m 

Category 9 is used to reflect student initiated questions or ideas. 
The subcategory 9^ indicates a student initiated conversational question 
or comment. The rare occurrences of this category were generally in 
English. The subcategory 92 indicates a student initiated question about 
the target language. It also includes unexpected student responses 
which generally were of a negative nature* that is, asking for repetition 
or clarification. 
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Examples of category 9^: 

"Why does Barbara say that Jean-Plerre is ' gelant'?" 

Examples of category 

"Que veut dire le mot ..." 

"Je ne comprends pas." 

Category 10 is used to indicate general silence or confusion. 

Inspired by Moskowitz, a certain convention was agreed upon in the coding 
of names. Names directly following a drill stimulus, a question, or a 
command were not coded separately. Names following an interval were 
coded separately in the same category as that which preceded, e.g., as 
a question if preceded by a drill cue, as a 4^ or if preceded by 
a question, or as a 6^ if preceded by a command. Names which preceded 
a stimulus, question, or command were coded separately in the same 
category as that which followed. Names in isolation were generally 
coded as 1 reflecting encouragement to a student volunteer. 

Idiosyncratic habits of a particular teacher (such as the continual 
usage of such phrases as "Bon" or "Alors") were coded separately and in 
the same category as that which followed if noticeably distinct. Such 
occurrences were not coded separately if they blended in with that 
which followed. 

A lecture sequence predominantly in Englisl , with occasional phrases 
in French, was coded as in L.. (English). Sentences half in English, half 
in French, were also coded all in L-. That is, there was a tendency to 
penalize even slight shifts into English. 

In addition to these categories reflecting teacher-student inter- 
action, the 0S1 system was further marked for emphasis and mode. Emphasis 
in the material presented by each teacher could be identified as 
neutral, grammatical, semantic, phonological, or other, e.g., cultural. 

It was anticipated that such information would indicate those areas 
which were of particular concern to each teacher, perhaps even those 
within which each teacher felt most comfortable. The possibility of 
dividing the material taught into distinct units could also offer an 
opportunity to compare more closely specific activities as handled by 
different teachers. It could also afford easily accessible information 
concerning the percentage of time spent by each teacher on a particular 
type of activity. 

The specification of mode was also deemed necessary in order to make 
possible a distinction between the use of the foreign language (L«) and 
English (L^). The possibility of distinguishing among speaking, reading, 
and writing in L- was also included as these modes would not be reflected 
in the use of the categories alone. The specification of reading behavior 
was considered important to qualify student responses which were not 
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freely spoken. Likewise the specification of writing behavior (by the 
teacher at the blackboard) was deemed necessary to qualify certain 
occurrences of the tenth category (silence or confusion). This distinc- 
tion among speaking, reading, and writing, however, did not prove 
particularly fruitful as it was difficult to code from tape and did 
not, in fact, need to be used very often. 

Provision had also been made to identify the teachers' use of 
audio-visual aids. As the use of such aids was almost entirely absent 
from the classes observed during this study, however, such a classification 
of behavior did not prove to be applicable and was eliminated. 



OS1 Categories for Foreign Language Teaching Interaction Analysis 





i. 


Comment 


1 

1 


Accepts feelings, jokes, makes comments that release ten- 
sion, uses students ideas. Tension releasing laughter. 








X 2 


Praises extensively - Encourages. 




2. 


Response 


2 1 


Gives short words of praise. 








2 2 


Reinforces student's response, using the same words, 
provides final model. 


PC 


3. 


Model 


3 1 


Gives model (first time or repeated), provides examples. 


i 






3 2 


Gives stimulus for drill. 


N 


4. 


Probe 


4 1 


Asks a conversation question. 








4 2 


Asks questions about the target language. 




5. 


Lecturing 


5 1 


Gives facts or opinions. 




6. 


Direction 


6 1 


Gives directions, ordrrs, or indications about procedures. 




7. 


Correction 


7 1 


Shapes behavior - facilitates student's answer by 
providing clues. 








h 


Rejects - Negative jokes. 




8. 


Response 


£ 

h 


Individual repetition. 








8 2 


Choral repetition or answer. 


i 






8 3 


Answer. 




9. 


Initiation 


9 1 


Asks a conversation question. Initiates a sequence. 








9 2 


Asks a conversation question about the target language. 




10. 




°1 


Silence or confusion. 
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Eac.i of the above categories is further marked for emphasis and mode. 



Emphasis 


Mode 


1 No particular emphasis 


l >2 Speaking 


2 On grammar 


L 2 Writing 


3 On meaning 


L 2 Reading 


4 On phonology 


L i 


5 On other aspects 


Jr/Mm 



Description of the Second Observational System (0S2) 

Foreign language instruction, especially in its beginning stages, 
differs radically in at least one important respect from all other types 
of instruction when the teaching is conducted in the foreign language: 
the medium of communication in the classroom is also the object of in- 
struction. The intellectual content conveyed through language is kep'. 
to a minimum: rather the main efforts of the students are geared towards 
the recognition and production of the forms and structures of the 
foreign language. Formal attributes of language can be described by 
a category system. It was assumed that, at the level of the sentence, 
a few of these attributes, well formed versus faulty, complete versus 
incomplete, type of syntactic structure, would be important factors in 
determining the quality of instruction and consequently its product, 
student learning. 

Thus the second system used (0S2) was designed to reflect the use 
of language in the classroom and the teacher's command of the foreign 
language. The unit of scoring was the sentence , and because sentence 
structure is usually very simple in the beginning classes, it was anti- 
cipated that decisions regarding this unit would not be impossible to 
reach. 

The sentence itself was defined operationally rather than linguistically. 
Incomplete sentences were those which the speaker intended to finish but 
either did not or could not. However, an utterance was classified as a 
cue if it were intended to be fragmentary, as in a drill situation. 

Complete sentences included both those which were grammatically complete 
and those which were functionally complete, that is, which had a sentence 
intonation. Teachers' sentences could also be categorized as non-well- 
formed . categories 2^ to 2^, in which case the complete-incomplete dis- 
tinction was ignored. 



COMMAND OF LANGUAGE 
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OS2 Categories for the Analysis of Teachers' Command of the Language 





Statement or 
answer 


Yes/no question 


QU question 


Order 

! 


1 

Other 


i 


2 


3 


4 


5 


X 1 


L 2 


Sentence by Teacher, correct 


complete 












X 2 


L 2 


Sentence by Teacher, correct 


repeated 












h 


L 2 


Sentence by Teacher, correct 


incomplete 












*^40 


L 2 


Cues, short praise, word, or word 


group 
















shaping by Teacher 
















L 2 


Sentence by Teacher, faulty 


two or more 


















error types 












2 2 


L 2 


Sentence by Teacher, faulty 


grammar 




ro 


L 2 


Sentence by Teacher, faulty 


lexis 




2 4 


L 2 


Sentence by Teacher, faulty 


phonology 




2 5 


L 2 


Sentence by Teacher, faulty 


repeated 




3 10 


L 2 


Sentence, by Teacher AND Students 














4 10 


L 2 


Sentence by Student (s) 


complete 












ro 

o 


L 2 


Sentence by Student (s) 


incomplete 












5 10 


L 1 


Sentence by Teacher 














5 20 


L 1 


Sentence by Student (s) 
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French phrases embedded within an English sentence were not coded 
separately unless they could be interpreted as distinct citations due 
to syntax or intonation. In practice, the coder was rather lenient 
with faults of intonation and minor distortions in articulation. As 
the distinction between correct and faulty would have been difficult 
to make in the case of student sentences, few sentences being 
acceptable as phonological correct, only the less subtle but more 
easily identifiable contrast between complete and incomplete was 
retained. 

Eight of 14 categories were secondarily marked for syntax (types 
of sentence structures). These were all the categories describing the 
teachers' use of the foreign language (except category I^q which was 
used to signal drill or shaping cues, usually in the form of a word or 
a word group, and isolated words of praise which were not seen as repre- 
sentative of the teachers' use of French). 

Experimental Conditions and Description of the Sample 

The present study was conducted in a university setting for a number 
of reasons. It was, first of all, the easiest and least costly alternative 
available. More important, however, instruction in the first-year 
language classes in the Romance Languages Department at the University 
of Michigan is highly regulated, permitting control of a number of 
variables not so easily possible elsewhere: a) all teachers involved 
are graduate students with comparable backgrounds and qualifications; 

b) the teachers involved have had the same prior training, i.e., a 
three-day workshop at the beginning of their teaching assignment; 

c) the teachers use the same materials and teach parallel classes 
according to the same fixed schedule, and the maintenance of an adequate 
teaching record reflecting the recommended methodology is the condition 
of their continued appointment; d) departmental examinations and 
standardized grading procedures are used to assess student performance; 
e) the students in each class are drawn from a homogeneous university 
population. 

In the sample used, age and sex of students were tested by Chi-square 
and proved to be statistically insignificant. Apart from the initial 
course taken in the department, previous exposure to French was nonexistent. 
It was impossible, however, to standardize class size (between 10 and 20 
students). It was discovered later that the relevant information 
necessary to carry out an analysis of covariance using the mid-term exam 
as the dependent variable and verbal SAT and the standardized departmental 
final examination at the end of the first semester course in French as 
covariables could not be obtained for all students. The original sample 
of 151 students enrolled dropped to 109, with a range of 7 to 16 per 
teacher. 
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Class sessions were distributed throughout the day. Likewise it 
was also not feasible to control for differences in motivation among the 
individual students , uor for differences in the amount and the quality 
of the homework demanded of the students. However, given the number 
and importance of controlled variables, it was hoped that differences 
in student results would be attributable largely to variations in 
teacher behavior. 

Original estimates of a study based on 20 teachers had to be 
modified due to a lower student enrollment than anticipated. Observa- 
tion of 11 teachers was actually begun, with the eventual loss of one 
teacher for personal reasons. The 10 remaining teachers included 5 
males and 5 females. Their teaching experience ranged from one to 
three years. 

Observation of Classes and Coding Process 

Early in the semester -a meeting was held with the department chair- 
man and the teachers to explain the general purpose of the research. 

It was emphasized that the study would not result in a departmental 
evaluation of individual teachers. They were not informed, however, 
of the exact nature of the category systems to be used in describing 
their classroom behavior. 

Soon after this preliminary meeting, the observers made at least 
one visit to each class to familiarize students and teachers with the 
taping and observational procedures to be used. The actual taping for 
coding purposes took place a few weeks later, after teachers and students 
had had an opportunity to become accustomed to each other, yet early 
enough in the term that they would not feel under undue pressure from 
approaching exams. The taping of four class sessions per teacher was 
carried out with in one week's time (the 2nd week of February) in order 
to capture a complete teaching cycle for each teacher. These four 
classes constituted a complete unit of instruction, the same for all 
ten teachers, during which they performed the whole range of activities 
required from them at this stage. Since all the teachers were using 
the same textbook, LlEchalle (Carduner, Hagiwara, Politzer, & 

Blaisdell, 1966) the content of those lessons was the same for all. 

The observers were present in each class and made some observations 
as to the classroom routine, timing, and teacher personality, according 
to previously prepared observation sheets. 

Originally a portable data recording device, the Multiplexor, was 
to be developed and tested to permit live observation under normal class- 
room conditions. Two observers, using two different category systems, 
could have used the same machine which would have punched both tallies 
and time intervals onto paper tape readable by a computer. This machine 
would have eliminated the costly and time consuming processes involved 
in the separate operations of observing, coding and storing, normally 
required in observational research of this type (see description of 
Multiplexor in Appendix 1). 
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As a result of delays In approval for purchase at the Office of 
Education, the Multiplexor was not available for in-class observation. 

This obliged the observers to code from audio tape recordings, lor 
this purpose, a coding box with ten buttons was built, and the CONCODE 
program for CATTS was adapted for each system to relay the coded data 
to a PDP-4 computer. This yielded two kinds of output: a punched paper 
tape and a print-out of the actual tallies. The data from the paper 
tape were later transferred to punched cards for analysis using an 
IBM 360-67 computer. 

Findings on Reliability 

Coding process . The data, originally tape-recorded, were coded 
within days of the actual observations. The coders had observed the 
classes and filled out a questionnaire including a breakdown of activities 
and comments on the teachers' performance, command of French, personality 
and on the organization and atmosphere of the class. These reports 
proved very helpful in the interpretation of some class sequences. 

All 40 class hours of tape were coded twice: once using 0S1, and 
the second time 0S2. Two coders were trained in the use of each 
system. Coders 1 and 3 were the principal coders for 0S1 and 0S2 
respectively, with Coders 2 and 4 serving to check the reliability of 
the principal coders. The same coder functioned to check reliability 
for 0S1, and as the principal coder for 0S2. 

All the coders had been thoroughly trained prior to the beginning 
of the experiment as they had conceived and perfected the coding systems 
over a period of three months. Numerous consultations between the 
coders took place during the actual coding process. The coding of 0S2 
was done by native speakers of French. All coders were well trained in 
linguistics . 

Although the two systems use low-inference categories (categories 
requiring minimal judgment on the part of the coder) , some problems of 
interpretation did arise (see earlier comments) . - But the main source 
of difficulty was the speed required for coding 4 digit tallies in 0S1. 

On the whole, the coders succeeded in coding in real time (at the pace 
of the class) but the timing was not deemed accurate enough to be used 
as a variable and further analyzed. However, it was a valuable indicator 
when sequences of tallies were compared to decide if two particular 
tallies coded at the same point in time were the same (although the 
preceding tally was missing in one of the sequences) or different. 
Comparisons were made by examining a computer print-out of the timed 
sequence of tallies, and when in doubt, playing back the original tape. 

Sampling of Data for Checking Reliability (Agreement) 

At the end of the principal coding task, portions of tapes for each 
teacher were selected at random. For 0S1 200-tally chunks were chosen 
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for recoding by the principal coder (intra-observer stability) and by 
a second coder (inter-observer agreement). For 0S2, 200-tal.'.y chunks 
were chosen for recoding by the principal coder, and 5 whole tapes 
were randomly chosen for recoding by Coder 4. 

Definition of Reliability 

Reliability was defined as the percent agreement between the number 
of tallies coded identically and the total number of tallies in the 
reliability sample. 

Misclassifications (i.e., differences between tallies for the 
same event) and omissions (i.e., missing tallies in one count) were 
counted as disagreements, with proper sequencing carefully observed 
except for inter-observer agreement in 0S2, where the results are 
reported in Table 1. As can be seen, very high levels of agreement. 



Insert Table 1 about here 



with little variability, were obtained in all Systems X Observers 
classifications. This is an important prerequisite to be satisfied for 
the analysis of the data collected in this research, since it implies 
that the classroom behavior was successfully rendered discrete, with 
highly systematic assignment of categories to behavioral events. 

Findings on the Frequency Analysis of the Systems 

For both frequency and Markovian analysis, the data were analyzed 
by a contingency table program written by Daniel J. Fox of the 
Statistical Research Laboratory of the University of Michigan, based 
on a minimum discrimination information statistic (Kullback, Kupperman 
& Ku, 1962) . This program computes an analysis of information table 
containing likelihood ratio statistics, degrees of freedom and attained 
significance levels 

a. that all pairs of variables are independent, 

b. that all three variables are independent, 

c. that two of the variables are independent of a third, 

d. that two of the variables are independent given a level of 
the third. 

In these tables, the probability that the variables considered are 
independent is abbreviated by P-, the probability level p varies from 
0 (certainly that they are related) to 1 (complete independence) ; df 
indicates the number of degrees of freedom for the test, which is equal 
to (C-l)(r-l), where c is the number of levels of the columns variable 
and r is the number of levels of the rows variable. Information 
indicates the value of the test statistic -2 In X, which Is asymtotically 
equivalent to x > component indicates the two variables for which the 
test is made. 
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In the following two subsections , the findings for the frequency 
and Markovian analysis of 0S1, and the frequency analysis of 0S2, are 
reported to show how the data were distributed for each variable in 
the systems, and which variables were found independent. 

1) 0S1 

A) There are 10 two-way tests of independence that can be computed 
on the frequency matrices for the five variables of 0S1. Of these, four 
are of no concern here, since they include classhour visited as one of 
the two variables in the test. The remaining six tests are reported in 
Table 2, and, for each, the full frequency matrix is reported in Tables 3 
to 8. Five of the six tests gave values significant beyond p » .001. 



Insert Tables 2-8 about here 



B) To determine which levels of Category interacted with Teachers, 
Category, with 18 levels, was successively recoded 18 times, to compare 
each level with the sum of all others. For each recombination, the 
statistic for Teacher x Category was then retested. The results are 
shown in Table 9. Seven of the 18 macrocategories were significantly 
discriminated among teachers in the sample at p < .05, with p < .001 
in all cases but one. 



Insert Table 9 about here 



For the 10 macrocategories in 0S1, the statistic for independence 
with Teachers was Information ■ 268.88, df * 81, p < .001. To determine 
which levels of Category in this sense interacted with Teachers, 

Category, now with 10 levels, was successively recoded 10 times, to 
compare each level with the sum of all the others. The statistics for 
independence for each recombination of Teacher x Category are shown in 
Table 10. Six of the 10 macrocategories were significantly discriminated 
.among teachers in the sample with p < .003. 



Insert Table 10 about here 



C) When Mode was recoded as French (1, 2, 3) vs. English (4), the 
test statistic rejecting independent with Teachers was Info » 11.58, df » 
9, p < .001. 

D) All levels of emphasis were found to interact significantly 
with Teachers, with p < .003. for each. 
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E) Microcategories 2 % 3, 6, 7, 8, and 10, and macrocategories 3, 

6, 7, 8, and 10 were found to interact significantly witl Mode, with 

p < .02 for each. 

F) No micro- or macrocategories interacted with Emphasis. 

G) With Mode recoded as French (1, 2, 3) vs. English (4), it was 
found to interact only with Emphasis 5 (other aspects). 

H) Finally, the 0S1 frequency data for categories were analyzed 

by the three-way contingency table program as one-step Markov chains with 
Teachers as a third dimension. The grand matrix for all teachers together 
on the Category variable is shown in Table 11. The individual matrices 
are included in Appendix 2 



Insert Table 11 about here 



Table 11 Category x Category in 0S1 

A three-way statistic of Teacher independence of both "preceding 
and succeeding" categories was calculated; however, there is reason to 
believe that these data .were not successfully analyzed by the program, 
and will have to be reanalyzed. 0 

A) There are three meaningful two-way tests of independent which 
exclude class hour visited which can be computed on the 0S2 variables. 
These are Teachers x Categories (14 levels) , Teachers x Syntax (5 levels) , 
and Syntax x Categories (8 levels). The results of these tests are 
shown in Table 12. 



Insert Table 12 about here 



The full frequency matrices for Teacher x Category, Teacher x 
Syntax, and Syntax x Category are reported in Tables 13, 14, and 15. 



Insert Tables 13 - 15 about here 



B) To determine which (if any) levels of Categories interacted 
at all with Teachers and Syntax, Categories were recoded into 14 two- 
level variables, using the same procedure followed in analyzing 0S1. 
The results are shown in Table 16. 



Insert Table 16 about here 
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C) Categories were also recoded to make specific Category x Teacher 
tests of independence: Teacher frequency (1. to 2_ + 3. n 

-x-i r J - ~ • c \ t-L i 



X XV + 5 10 > V8, 

the other frequencies (3 in to 4„ n + 5 9n ), L 9 by Teacher (1. to 2,.) vs 
L by Teacher (5 ), L By StudeKts (3^ 4 * 4 ) vs. L,*"' " ^ - 

(Sjq), and Correct Sentences by Teacher (1., 1 9 , I«) vs. 

Sentences (2^ to 2^) by Teacher. The results are reported in Table 17 



by Students 
Incorrect 



Insert Table 17 about here 



D) To determine which levels of Syntax interacted significantly 
with Teachers, the variable Syntax was recoded into five two-level 
variables comparing each level with ail other levels. The results 
are shown in Table 18. All levels were significant beyond p *= .001. 



Insert Table 18 about here 



Testing of Hypotheses 

A) Criterion measures of teacher effectiveness 

Classroom achievement scores . Students' performance on the Verbal 
Scholastic Aptitude Test (VSAT) of the College Entrance Examination 
Board (CEEB) tests, and on a standardized achievement test for first 
semester college French used in the Department of Romance Languages, 
were used as covariates in the evaluation of students' performance on 
the second semester mid-term exam. This test was administered one 
week after the end of classroom observations and yielded scores on two 
dependent variables: one for the total exam (MID), and one for only 
the first part based on oral comprehension (MIDI), which was expected 
to be a closer reflection of effects of the type of teaching previously 
observed (see Appendix 3). The number of students in this analysis (for 
whom all of the relevant data were available) was 109, ranging from 7 
to 16 across the 10 teachers. In the analysis of covariance, raw 
scores were transformed to logarithms to correct for positive skew, 
and the equality of slope assumption of the test: for MID, f (9,97) « 
1.91, .05 < p < .10, and for MIDI, f (9,97) « 3.28, p < .01. The 
geometric means based on adjusted group scores are shown in Table 19. 



Insert Table 19 about here 



Only one teacher (S.S.) was shown by post hoc t-tests to be signif- 
icantly different from most other teachers, and this presumably accounted 
for the achieved levels of significant. The original purpose of the 
analysis, to determine statistically groups of high-achieving and low- 
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achieving teachers, was thus frustrated. Instead of testing hypotheses 
about teacher effectiveness by dividing the sample into two groups and 
using t, or A 2 , Spearman rg correlations based on the ranks of adjusted 
group means and those of ratios of category use were employed in the 
predictive analyses. 

Student ratings . At the end of the term a questionnaire was given 
to all the students at the same time. This is a standardized procedure 
in the department. The first 17 questions on a 7-point scale ranging 
from below to above average, provide the basis for an evaluation of each 
instructor (see Appendix 3) . The number of students answer all 17 
questions was 123 with a range of 7-15 per teacher. Mean ratings for 
all 17 questions are presented in Table 20, along with the grand means. 



Insert Table 20 about here 



Observer ratings . Although to this date reported results provide 
only moderate support for the value of observer ratings as a measure of 
teacher effectiveness, the two principal observers who had filled out 
questionnaires (see Appendix 4) at the end of each class produced two 
independent lists of the ten teachers, ranking them in descending order 
on two aspects, command of French and quality of teaching techniques. 

Summary . The five evaluation variables, mid-term test (WMID) , first 
part of mid-term test (MIDI) student ratings, and two observers’ ratings 
are reported in Table 21. A correlation of each pair of evaluation 



Insert Table 21 about here 



variables in terms of the squared difference term in Rho (df = 9) is 
reported in Table 22. 



Insert Table 22 about here 



It can be seen that MID - MID I scores, the MID - students' ratings, 
and students - teachers' command of French were all positively correlated 
beyond the .025 confidence levels by 1-tailed Spearman Rho tests: MID - 
command of French and the two observer ratings were positively correlated 
beyond the .05 level. 

B) Hypotheses 

The following hypotheses were formulated according to current assump- 
tions about what constitutes effective teaching in a beginning language 
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class. It is assumed that their implications do not extend beyond the 
limits of the initial phase of language teaching. 

The ratios presented below using frequency data from 0S1, 0S2, and 
0S1 1-step tables, expressed in terms of the frequency of whole categories 
for 0S1, and 0S2 and of cell frequencies for 0S1 1-step, arc not empirical 
hypotheses themselves, but measures of behavior, the hypotheses are that 
these measures will be related to measures of effectiveness, i.e., the 
evaluation criteria. Thus, H. is a prediction about correlation between 
test scores and teacher's participation/students ' participation. 

The five evaluation measures provide five different rank ordered 
lists of the ten teachers in the sample. The ratios computed from the 
observed frequencies for each teacher will be rank-ordered each time . 
and compared with the criterion measures. So that stating in a 
hypothesis that the teacher getting the highest ratio, i.e., getting 
first rank in the ordered list will be most effective, also implies 
that the teacher getting the second rank will be the next most effective, 
etc. 



a) Hypotheses tested with ratios based on 0S1 frequency data. 

HI: A language teacher who speaks least and has his students speak 
most will be most effective. 



In terms of 0S1 categories, this H. was to be tested using the 
following ratio: 

cat. 1 to 7 _ T.'s total participation 
cat. 8+9 - £s' total participation 

and ignoring cat. 10, silence or confusion. 

H2: A language teacher who scores higher on categories construed 

as denoting "indirect influence" than on categories relating to "direct 
influence" will be most effective. 



Hn focuses on the attention given by the teacher to students' behavior 
and ideas, and his praise and reinforcement as contrasted with the teacher's 
directing and criticizing. 



1^ + I 2 + 2^ + 2 2 
6 1 + 7 1 + 7 2 



This hypothesis roughly corresponds, in terms of absolute frequencies 
of use, to what are known in the literature as "indirect" vs. "direct" 
influence" or "integrative" vs. "dominative" types of teachers (Flanders, 
1966). 
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H3: A language teacher who elicits students' participation and 

practice by using more drills and questions than lecturing will be 
most effective. 



In terms of 0S1 categories: H3^ was to be tested with the ratio: 



3 1 + 3 2 + 4i + 4 2 



This ratio was further refined for as: 



3 ^ + 3 2 + 4 ^ + 4 2 + 7 ^ 



to include 1 ) teacher shaping techniques of correction as a way to elicit 
student practice of the foreign language (7^) and 2 ) rejecting criticism 
(? 2 > as inhibiting students' participation. Although reflecting teacher 
type and degree of influence H3^ and H3 0 are more concerned with basic 
classroom techniques. 

H4: A language teacher who uses shaping techniques for correction 

will be more effective than the teacher who rejects students' incorrect 
responses. In terms of 0S1 categories, 7^, the greater the ratio the 




the more effective the teacher. This ratio can be considered as a simpler 
but less precise measure of the teacher's approach already measured in H^: 
solicitation vs. imposition. 

H5: A language teacher who encourages students' spontaneous participa- 

tion in the foreign language (I^) in preference to interventions in the 
native language (L^) will be more effective. In terms of 0S1 categories: 

9 ± + 9 2 in L x + L 2 
9i + 9j in alone 

This can be interpreted as a measure of the degree of willingness and 
spontaneity with which the students initiate sequences in the foreign 
language. 

H 6 : A language teacher who encourages his students to use the 
foreign language more spontaneously will be more effective. In terms 
of 0S1 categories: 

8 i + 82 + 8 ^ spontaneous contributions in 
9 + 9 2 conditioned responses in 

That is, the greater the ratio, the more spontaneous the students are in 
their use of the foreign language. 
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H7: A language teacher more restrictive of the use of the student's 

native language in the class-room will be more effective. In terms of 
0S1 categories: 



Modes 1, 2, and 3 (use of L 2 ) 

Mode 4 

H8: A language teacher more restrictive of the use of the native 

language by his students will be more effective. In terms of 0S1 categories 

8 1 + 8 2 + 8 3 + + 9 2 (L 2 ) 

8 1 + 8 2 + 8 3 + 9 i + 9 2 < L i> 

H9: A language teacher who obtains the more student participation 

in the foreign language — excluding mere repetition — will be more effective. 
In terms of 0S1 categories: 

total tallies 
®3 + 9 1 + 9 2 ^ L 2* 

HI assumes that the way talking time is distributed between T and Ss in 
the classroom will affect efficiency. 

H2.3, and 4 assume that the use of indirect influence coupled with 
eliciting techniques will produce better learning than direct influence 
and lecturing. 

H5 and 6 assume that a classroom climate and teaching techniques producing 
more spontaneous use of the foreign language by the Ss will produce better 
learning. 

H7,8, and 9 assume that maximal use of the foreign language in the class- 
room will produce better learning. 

b) Hypotheses tested with ratios based on 0S2 frequency data. 

The first three hypotheses assume that the proportion of sentences 
in the foreign language which are used by T and Ss will be correlated 
with learning. 

HI duplicates H7 in 0S1. 

A language teacher more restrictive of the use of sentences in the 
students' native language in the classroom will be more effective. In 
terms of 0S2 categories: 

cat 1 to 4 (I* 2 ) 

(Lp- 



cat 5 
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Hl^: A language teacher who uses the foreign language more will 
be more effective. In terms of 0S2 categories: 

* cat 1 to 3 (L2 T) 

cat~l 10 CL ± T) 

HI3 replicates H g in 0S1. 

A language teacher more restrictive of the use of sentences in the 
native language by his students will be more effective. In terms of 
0S2 categories: 



cat 3+4 (Lj, Ss) 
cat ^2Q 



H2. and H2 2 relate to the proportion of teacher and student talk 
in the foreign language and in both foreign and native language. 

H2 : A language teacher who speaks least in the foreign language 
and has nis students speak most will be more effective. In terms of 
0S2 categories: 1 to 3 (L 2 ) 

3 + 4 ” 1 



H2^ replicates in 0S1. 

A language teacher who speaks least and has his students speak 
most will be more effective. In terms of 0S2 categories: 

1 3 + 5 10 < L 1 +L 2> 



H3 and 4 assume that the greater the degree of "correctness" of 
the foreign language used in the classroom, the better will be learning. 

Correctness is computed in terms of correct vs. faulty sentences 
for the teacher (excluding cat. I^q which is mostly drill or shaping 
cues), and in terms of complete vs; incomplete sentences for the students. 

H3: A language teacher who uses the foreign language more correctly 

will be more effective. 



H4: A language teacher who makes his students produce a greater 

proportion of complete sentences vs. incomplete sentences in the foreign 
language will be more effective. In terms of 0S2 categories, the ratios 
are respectively: 



cat 1 
cat 2 



cat 

cat 
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H5 and 6 assume that the use of questions is an effective technique 
to develop _Ss' production and ability to use the foreign language as a 
real means of communication. 

H5: A language teacher who uses the greater proportion of questions 

will be more effective. 



H6: A language teacher who uses the greater proportion of QU 

questions will be more effective. In terms of 0S2 categories: 

Syntax (2nd subscript) 2+3 
Syntax 1 to 5 



Syntax 3 
Syntax 2 



c) Hypotheses based on 0S1 1-step frequency data. 

HI: Using Soar's definition of flexibility (how many cells are 
needed to account for 60 of the tallies) this hypothesis predicts that 
a more flexible language teacher will be more effective. 



H2: Using the Cartesian product of i (1. , lg, 2., 2^) by itself 
as a measure of extended indirect influence (I) and tne Cartesian 
product of d (6j, 7., 7,) ^y itself as a measure of extended direct 
influence (d), H2 predicts that the language teacher using propor- 
tionally more indirect influence will be more effective. In terms of 
cell frequencies: 



i 

d 6. 7 



1 2AA 



1 / 1, ? 2 



in same columns. 



H3: A language teacher who shows greater concern for students' 

participation will be more effective. The ratio is obtained by examining 
rows 8^ to 9_, i.e., the way teacher reacts to students' participation. 
Indirect influence and maintaining the exchange, categories 1 to 4 are 
contrasted with direct influence and lecturing, categories 5 to 7. In 
terms of cell frequencies: 



Rows 8. to 9, cat 1 to 4 

1 cat 5 to 7 



H4: The language teacher who uses drill stimuli and questions will be 

more effective, 
d) Results. 

The correlations with the five evaluation variables are reported 
in Table 23. The three tables of ratios by teachers are given in 
Appendix 6. Twenty-two correlations out of the 120 which were computed 
were significant, 19 beyond the .05 level of confidence, 2 beyond the 
.025 level and 1 beyond the .005 level. There were 17 positive and 
5 negative correlations. 



ERIC 



d O 



Capelle, Jarvella, & Revelle 



26 



Insert Table 23 about here 



None of the ratios correlated with the second evaluation variable , 
the first part of the mid-term exam. There were four positive and one 
negative correlation with variable 1, adjusted mid-term scores, five 
positive and two negative correlations with variable 4, observers’ 
ratings of teachers’ command of French and two positive correlations 
with variable 5, observers’ ratings of teachers’ techniques. 



1) Of the 50 correlations using ratios based on 0S1 raw frequencies 
only two were significant, both involving variable 3, students’ ratings. 
It appears that students are sensitive to the type of criticism used 
by the teacher (shaping vs. rejection). The correlation with the ratio 



2 was significant beyond the .025 level of confidence. Also, students’ 
ratings must have been influenced by the proportion of foreign vs. native 
language used in the classroom as the ratio for H7 correlated positively 
with variable 3. 



2) Of the 45 correlations using ratios based on 0S2 frequencies 
nine were significant. 



HI. predicted that effectiveness would be correlated with the ratio 
of the total number of sentences in the foreign language to the total 
number of sentences in the native language. The ratio was negatively 
correlated with variable 4. In fact, the teachers whose French was most 
correct were not those who used French most in class and this may account 
for this finding. The observers did not take into account the sheer 
amount of French used in the classroom but the teachers' command of French. 



HI,, predicted that effectiveness would be correlated with the ratio 
of foreign to native language sentences used by the students. The ratio 
was negatively correlated with variables 1 and 4. The sheer amount of 
sentences used is not the important factor here, but the quality of the 
French used. 



In those three hypotheses predictions should not be based on the 
numbers of sentences used in the and but on the quality of the 
language used, given a sufficient number of sentences. 

H2^ predicted that effectiveness would be correlated with the ratio 
of the number of sentences spoken by the teacher to the number of 

sentences spoken by the students. This was parallel to H7 in 0S1. 

There was a positive correlation with variable 5, observers' ratings of 
teachers ’ techniques • 
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The best predictor was H3. The degree of correctness of the teacher 
in the foreign language was positively correlated with 4 of the five 
evaluation variables, with variable 1 beyond the .025 level of confidence 
and with variable 3 beyond the .005 level. This talent of the teacher 
seems to be well perceived by the students and may Increase their con- 
fidence In him. 

H4 predicted that students producing the greater proportion of 
complete, vs. Incomplete, sentences would learn more and consequently 
that the most effective teacher would make their students produce complete 
sentences whenever possible. The ratios correlated negatively with 
variables 1 and 3. 

Again quality of the language used and classroom situation should 
be considered here. It was observed that. In some classes, students 
practiced build-up reading thus producing a number of Incomplete sen- 
tences In a situation where this practice could not be construed as 
detrimental. This hypothesis must be abandoned as It Is not possible 
at this point, using this system, to differentiate between types of use 
and teaching situations. 

The greater the ratio of QU questions over "Oui - si - non" 
questions, the more the observers seem to be Influenced In their evalua- 
tion of the teachers' command of French. 

3) Of the 25 correlations using ratios based on the 0S1 Markovian 
matrices eight were positive and one negative. 

Flexibility defined after Soar (1966) as the number of cell frequencies 
accounting for 60% of the total number of tallies correlated with variables 
1, 3 and 4. This seems to be an important dimension of teacher effectiveness. 

i_ 

The ratio of extended d correlated with variables 1, 3 and 4 thus 
confirming the hypothesis that sustained indirect influence makes for 
greater effectiveness than sustained direct influence. But this global 
result will have to be further tested in sspecific classroom situations 
as it is suspected that indirect and direct influence can have various 
effects depending on the degree of clarity of the goals for the students 
(Flanders, 1966). 

Another measure of 4 , computed from the reactions of the teachers 
following student talk yielded a negative correlation with variable 3. 

This hypothesis will be reformulated to eliminate 7j, shaping, which 
may not be felt as direct influence by the students but as a necessary 
help. 



H4 and 5 were sustained by one positive correlation each with variable 
4. This seems insufficient as the prediction was based on the expectation 
that certain techniques were more effective, thus bearing little relation 
to the teachers' command of French. 
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1) Description power of the systems. 

A. The major purpose of this research was to determine the descriptive 
power of two specially developed observational systems. It may be illus- 
trative to present the profiles of the average teacher and students as 
performers in the language classroom. The following sketch, based ex- 
clusively or. statistical data to the exclusion of the comments recorded 
by the observers, does not exhaust all the possibilities of interpreta- 
tion as it describes only the average teacher and students. The descrip- 
tion applies only to the . classes observed and should not be 

construed as a prescriptive model of what a beginning language class 
should be at the college level. 

1. Distribution of activities as observed through verbal behavior . 

The teacher is the most active as he performs 2/3 of the units of 
behavior (66.5%) whereas the students perform 24.5%. Silence is limited 
to a minimum (9%) and occurs mostly when students hesitate before 
answering. Sixteen percent of all verbal behavior is in the students' 
native language and 84% in the foreign language, oral practice accounting 
for 74% and reading for 9%, the remaining 1% being taken up by writing, 
usually by the teacher on the blackboard. 

More than half of the total time is devoted to grammar (52%) drills 
or explanations and the 36% described as "general use" are heavily 
fraught with grammatical intentions on the teacher's part. Ten percent 
of the total time is devoted to explanations of meaning and half the 
time these explanations are conducted in English. Phonetics is restricted 
to 1% of total activities and about a fourth of this time (.225%) is 
taken up by explanations in English. The teaching of culture was not 
a primary aim of this language course: explicit focus on culture occurs 
less than 1% of the total class time. 

The typical class was 48 min. long. 

2. Teacher profile . 

Teacher use of language . On the whole the teacher uses French with 
a high degree of correctness (91%). Nine percent of all teachers 1 sen- 
tences are faulty, 27.5% of the errors being grammatical and 61.5% 
phonetic. Only rarely (2%) does the teacher produce a sentence contain- 
ing two or more mistakes. 

Statements account for 47% of all the sentences uttered by the 
teacher (excluding drill cues), questions for 23.5%, orders for 19%. 
Approximately one sentence out of seven is intentionally repeated, and 
one out of six utterances is a cue. The teacher uses proportionally 
more English than his students: 21.5% of all teacher utterances against 
15.5% of all student utterances are in English. 
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Teacher techniques . The teacher appears primarily as a lecturer 
since nearly one-third of his total verbal behavior is in category 5 
(lecturing and explaining), and 38% of his lecturing is done in English. 

On the other hand he is little prone to using sharp criticism (1%) , 
nor does he readily accept student's feelings, use their ideas, or 
release tension (1%). 

Nearly one-third of his time is occupied in modeling and probing 
(31.5%). These activities include producing models'for imitation (9.5%), 
giving drill cues (6.5%), asking conversational questions (10%), asking 
questions about the language taught (5.5%). One fourth of the questions 
asked about the language are asked in English. 

The teacher gives most of his directions in French and this takes 
up 12% of his moves. 

He confirms students' answers by repeating them (10% of his moves). 
Most of the time confirmation is accompanied by short words of praise 
(6.5% of total teacher moves) and extended praise (3%). 

If rejecting criticism is rare, shaping (5% of total teacher moves) 
is used mostly for grammar and for phonetics. 

3. Student profile . 

The student is primarily a respondent. Nearly half of his moves 
(46%) are individual responses. More than one-f outfit (27%) are choral 
repetition or more rarely, choral responses to drill stimuli, 17% are 
devoted to individual repetition. 

He seldom initiates a sequence (categories 9^ and 92 account for 
2% of total classroom units of behavior) and When ne does, he does so 
in English two- thirds of the time. 

Fifteen and a half percent of his utterances are in English and 
he produces two complete sentences for each incomplete one. 

B. The goals of the present research as outlined in the introduc- 
tion have been largely attained. Both major observational systems 
developed were applied with very high observer agreement to a sub- 
stantial amount of classroom behavior. The systems' independent variables 
of categories (and their hierarchical organization) , language used (French 
or English) and its sentence structure (French) , teaching emphasis and 
teachers were found to interact in significant ways over this discretely 
resolved behavior. In particular, for 0S1, many of the categories were 
found to vary with even the relatively homogeneous sample of teachers 
participating in the experiment, and hypotheses for effectiveness based 
on Markovian analyses were generally confirmed. For 0S2, many of the 
hypotheses described were also confirmed, even using the relatively weak 
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nonparametric test of correlation between derived category ratios and 
estimates of teacher effectiveness. Thus, the present research csn be 
said to have succeeded at least in part in both aspects of the goals 
outlined: we have described and quantified foreign language classroom 
behavior, and succeeded in correlating it with both objective and more 
subjective measures of effectiveness. While there is much in the data 
that will be refined in still further analyses, at least this much can 
be said to already have been accomplished. 

2) Implications for future research 

The present research has been considered as only the initial step 
in a long-term project. The observational systems used, although care- 
fully developed and fulfilling the major requirements for category 
systems intended for this use, will need to be modified in various ways 
to answer some of the questions which have arisen over the analysis of 
results. A second research effort which will need to be undertaken 
before the systems ecu with reasonable confidence provide criteria for 
effective teaching, and for teacher training, is a correlational study 
employing a wider range of instructors, in order to estimate what the 
range and distribution of varous behavior measures are like, how each 
measure is related to the most objective criterion for effectiveness 
attainable under these less controlled circumstances, and to retest 
the applicability and exclusive nature of the systems themselves. To 
do otherwise, and nevertheless make concrete applied proposals for 
training, would be to prejudge the issues involved favoring the better 
teaching techniques in the study, and extend without empirical verifica- 
tion interpretations based on its possibly biased sample of behavior. 
Further research on observational systems in foreign language teaching 
can profit greatly from this initial attempt to apply the current 
observational systems. But, like other experimental findings, the 
present findings can be extended beyond the scope of the study only in 
the most cautious and tentative way. 



Footnotes 

^The research reported herein was supported in part pursuant to 
Contract 0EC-0-9-097740-3743(014) with the U. S. Department of Health, 
Education, and Welfare, Office of Education, under the provisions of 
P. L. 83-531, Cooperative Research, and the provisions of Title VI, 

P. L. 85-864, as amended. 
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Table 1 

Observer Agreement in Coding Classroom Behavior 



Kind of 


Observation System 


Agreement 


0S1 


0S2 


Inter-observer 


X * 87 


X - 92 




SD = 3.7 


SD - 2.6 


In tra- observer 


X = 94 


X - 88 




SD » 2.5 


SD = 3.7 



Table 2 

Two-Way Statistics for 0S1 



Component 


Information 


df 


P 


Teacher x Category 


2253.54 


153 


.001 


Teacher x Mode 


9470.94 


30 


.001 


Teacher x Emphasis 


7776.09 


36 


.001 


Category x Mode 


568.29 


51 


.001 


Category x Emphasis 


0.0 


68 


1.0 


Mode x Emphasis 


682162 


12 


.001 
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Table 3 

Frequency Distribution of Teacher by Emphasis in 0S1 



Empha 



Teach 


1 


2 


3 


4 


5 


Total 


1 


1533 


1711 


352 


88 


158 


3842 


2 


1749 


2566 


211 


99 


13 


4638 


3 


1180 


2501 


839 


17 


11 


4548 


4 


2223 


2039 


98 


19 


23 


4402 


5 


1593 


2575 


323 


75 


59 


4625 


6 


1611 


2011 


263 


32 


200 . 


4117 


7 


1878 


1655 


662 


16 


37 


4248 


8 


1441 


2523 


716 


37 


0 


4717 


9 


1322 


2680 


180 


131 


0 


4313 


10 


1632 


2747 


674 


46 


0 


5099 


TOTAL 


16162 


23008 


4318 


560 


501 


44549 



Table 4 

Frequency Distribution of Emphasis by Mode in 0S1 



Mode 

Empha 


1 


2 


3 


4 


Total 


1 


11114 


96 


3931 


1021 


16162 


2 


18881 


251 


0 


3876 


23008 


3 


2127 


64 


2 


2125 


4318 


4 


384 


11 


40 


125 


560 


5 


485 


2 


0 


14 


501 


Total 


32991 


424 


3973 


7161 


44549 
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Table 5 

Frequency Distribution of Teacher by Category In 0S1 



Te 
Cdt • 


1 


2 


3 


4 


5 


6 


7 


8 


9 


10 


Total 


i 


10 


17 


25 


48 


25 


15 


13 


12 


13 


39 


217 


2 


110 


88 


51 


88 


55 


84 


23 


162 


72 


111 


844 


3 


162 


188 


146 


249 


112 


296 


89 


287 


197 


195 


1921 


4 


177 


358 


413 


278 


340 


163 


234 


358 


248 


469 


3038 


5 


342 


116 


218 


508 


504 


120 


231 


302 


247 


164 


2752 


6 


95 


154 


267 


166 


277 


163 


163 


187 


256 


181 


1409 


7 


209 


228 


346 


361 


90 


240 


648 


199 


251 


465 


3037 


8 


130 


138 


183 


35 


69 


159 


221 


257 


75 


303 


1570 


9 


909 


1163 


1217 


569 


1181 


902 


784 


845 


764 


697 


9031 


10 


289 


568 


385 


299 


215 


355 


287 


366 


391 


344 


3499 


11 


75 


134 


118 


177 


125 


116 


68 


169 


293 


202 


1477 


12 


53 


31 


19 


8 


29 


6 


27 


31 


34 


24 


262 


13 


115 


247 


80 


264 


200 


303 


125 


181 


190 


185 


1900 


14 


310 


205 


179 


564 


618 


231 


191 


271 


171 


221 


2961 


15 


410 


490 


548 


390 


417 


383 


460 


561 


571 


808 


5038 


16 


66 


35 


39 


78 


36 


65 


36 


54 


54 


107 


571 


17 


26 


34 


30 


7 


58 


80 


16 


175 


54 


85 


515 


18 


354 


443 


284 


313 


274 


436 


622 


310 


432 


549 


4007 


Total 


3842 


4638 


4548 


4402 


4625 


4117 


4248 


4717 


4313 


5099 


44549 
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Table 6 

Frequency Distribution of Emphasis by Category in 0S1 



Empha 

Cat 


1 


2 


3 


4 


5 


Total 


i 


100 


85 


21 


2 


3 


217 


2 


305 


448 


76 


6 


9 


844 


3 


685 


1031 


182 


22 


1 


1921 


4 


730 


1947 


322 


34 


5 


3038 


5 


1464 


1208 


11 


69 


0 


2752 


6 


21 


1886 


2 


0 


0 


1909 


7 


2361 


493 


161 


0 


22 


3037 


8 


61 


800 


698 


11 


0 


1570 


9 


2339 


4727 


1407 


144 


414 


9031 


10 


1366 


1943 


141 


46 


3 


3499 


11 


612 


729 


80 


56 


0 


1477 


12 


74 


130 


55 


3 


0 


262 


13 


1473 


370 


3 


54 


0 


1900 


14 


968 


1918 


2 


73 


0 


2961 


15 


1504 


2929 


592 


7 


6 


5038 


16 


314 


198 


50 


2 


7 


571 


17 


9 


407 


74 


22 


3 


515 


18 


1770 


1759 


441 


9 


28 


4007 


Total 


16162 


23008 


4318 


560 


501 


44549 
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Table 7 

Frequency Distribution of Teacher by Mode in 0S1 



Mode 

Teach 


1 


2 


3 


4 


Total 


1 


2489 


93 


437 


823 


3842 


2 


3387 


17 


421 


812 


4638 


3 


2583 


25 


166 


1774 


4548 


4 


3464 


34 


841 


63 


4402 


5 


4110 


17 


175 


323 


4625 


6 


3195 


39 


537 


346 


4117 


7 


3724 


104 


378 


42 


4248 


8 


2434 


14 


565 


1704 


4717 


9 


3230 


2 


262 


819 


4313 


10 


4375 


79 


191 


454 


5099 


Total 


32991 


424 


3973 


7161 


44549 
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Frequency Distribution of Mode by Category in 0S1 
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Table 9 

Two-Way Statistics for 18 Microcategory x Teachers Tests in 0S1 



Component Information df p 



Cats. 1^1 $ 2 » ^2* 

cat . ^2 

cat • 2^ 

cat. 22 

cat. 3^ 

cat. 4^ 

cat. 5^ 

cat. 6^ 

cat. 7^ 

cat . ?2 

cat. 8^ 

cat . 82 

cat . 3 j 

cat. 9^ 

cat. 10 ^ 



92 0.0 each 

4.68 
12.48 
18.84 

94.96 
40.01 

24.97 
10.18 
77.15 

5.11 

221.75 

178.76 
2.55 

11.72 

3.22 



9 each 

9 

9 

9 

9 

9 

9 

9 

9 

9 

9 

9 

9 

9 

9 



1.0 each 
.86 
.19 
.03 
.001 
.001 
.001 
.335 
.001 
.82 
.001 
.001 
.98 
.23 
.96 
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Table 10 

Two-Way Statistics for 10 Macrocategory x Teacher Tests in 0S1 



Component 


Information 


df 


P 


Cat 1 


4.68 


9 


.86 


Cat 2 


25.80 


9 


.003 


Cat 3 


94.96 


9 


.001 


Cat 4 


40.01 


9 


.001 


Cat 5 


24.97 


9 


.001 


Cat 6 


10.18 


9 


.34 


Cat 7 


76.49 


9 


.001 


Cat 8 


43.13 


9 


.001 


Cat 9 


11.72 


9 


.23 


Cat 10 


3.22 


9 


.96 
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Table 11 

Markovian One-Step Grand Frequency Matrix for 0S1 



Categories 





1 


2 


3 


4 


5 


6 


7 


8 


9 


10 


11 


12 


1? 


14 


1? 


16 


17 


18 


1 . 


30 


4 


7 


7 


7 


5 


27 


1 


50 


25 


1 


0 


1 


0 


8 


13 


3 


28 


2. 


14 


85 


5 


10 


0 


8 


29 


8 


60 


66 


5 


0 


12 


3 


275 


66 


113 


85 


3. 


20 


29 


35 


584 


89 


183 


138 


75 


259 


275 


24 


3 


42 


2 


46 


3 


3 


111 


4. 


17 


27 


573 


261 


31 


387 


240 


102 


501 


302 


16 


4 


100 


283 


65 


14 


8 


107 


5. 


3 


9 


3 


0 


663 


37 


28 


53 


86 


249 


5 


0 


98 


1442 


30 


0 


1 


45 


6. 


2 


7 


0 


7 


3 


409 


5 


20 


32 


73 


15 


0 


5 


338 


705 


26 


2 


260 


7. 


14 


57 


2 


9 


10 


3 1293 


19 


183 


27 


13 


2 


1 


3 


579 


76 


7 


739 


8. 


6 


40 


1 


6 


9 


6 


12 


492 


130 


24 


12 


1 


0 


4 


473 


16 


6 


332 


9. 


35 


191 


59 


138 


294 


111 


419 


317 


6107 


428 


37 


8 


21 


1 


132 


83 


137 


576 


10. 


13 


31 


5 


87 


359 


165 


97 


66 


211 


1182 


67 


5 


212 


401 


164' 


28 


7 


399 


11. 


0 


9 


16 


22 


6 


12 


25 


21 


84 


65 


70 


0 


400 


50 


605 


21 


7 


64 


12. 


1 


9 


1 


7 


7 


11 


21 


31 


52 


21 


40 


21 


1 


0 


27 


1 


1 


10 


13. 


3 


11 


183 


144 


88 


10 


7 


7 


17 


55 


369 


8 


966 


0 


5 


7 


1 


19 


14. 


9 


14 


274 


315 


1117 


304 


10 


56 


168 


182 


41 


7 


5 


412 


2 


0 


i 

j. 


43 


15. 


20 


118 


721 


1391 


8 


87 


89 


76 


116 


77 


671 


187 


0 


1 


1173 


21 


4 


278 


16. 


13 


30 


6 


12 


3 


43 


87 


8 


197 


45 


25 


0 


2 


0 


12 


59 


0 


29 


17. 


0 


20 


3 


6 


1 


4 


2 


4 


249 


8 


2 


1 


0 


0 


1 


0 


192 


22 


18. 


17 


153 


27 


32 


57 


124 


507 


214 


591 


452 


64 


15 


34 


20 


736 


137 


22 


860 



Table 12 

Tests of Independence for 0S2 Variables 



Component 


Information 


df 


Signif. 


Teacher X Category (14) 


9.858 


117 


.001 


Teacher X Syntax 


441.33 


36 


.001 


Syntax X Category (8) 


76.882 


28 


.001 
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Table 13 

0S2 Frequency Distribution of Teacher by Category 



Teach 


1 


2 


3 


4 


5 


6 


7 


8 


9 


10 


Total 


Categ. 
























1 


1263 


1316 


1386 


2024 


1443 


1525 


1621 


1228 


1096 


1842 


14649 


2 


261 


203 


147 


280 


415 


97 


134 


189 


184 


313 


2223 


3 


3 


25 


33 


15 


25 


22 


25 


1 


12 


26 


187 


4 


233 


883 


478 


327 


508 


242 


333 


483 


587 


534 


4308 


5 


3 


7 


1 


1 


1 


4 


16 


0 


1 


0 


34 


6 


26 


105 


52 


30 


50 


56 


32 


3 


24 


9 


367 


7 


12 


34 


3 


5 


19 


13 


15 


2 


6 


3 


112 


8 


33 


66 


62 


114 


84 


73 


319 


11 


50 


12 


824 


9 


9 


36 


15 


17 


63 


19 


105 


2 


88 


8 


332 


10 


0 


1 


0 


0 


0 


0 


0 


0 


0 


0 


7 


11 


476 


538 


402 


649 


634 


342 


418 


320 


395 


500 


4676 


12 


77 


110 


185 


215 


368 


195 


185 


254 


395 


353 


2287 


13 


404 


519 


1179 


35 


185 


105 


27 


913 


423 


250 


4044 


14 


123 


64 


119 


20 


87 


122 


19 


348 


78 


109 


1089 


Total 


2925 


3607 


3942 


3743 


3886 


2875 


3249 


3754 


3259 


3459 


33139 
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Table 14 

0S2 Frequency Distribution of Syntax by Teacher 



Synt 


1 


2 


3 


4 


5 


Total 


Teach 














1 


821 


83 


230 


337 


139 


1610 


2 


932 


131 


171 


390 


168 


1792 


3 


773 


113 


193 


365 


135 


1579 


4 


1197 


336 


304 


356 


298 


2491 


5 


1281 


163 


173 


237 


246 


2100 


6 


772 


218 


213 


255 


351 


1805 


7 


1003 


145 


523 


424 


122 


2267 


8 


549 


135 


280 


309 


163 


1436 


9 


548 


85 


148 


392 


208 


1431 


10 


924 


174 


469 


492 


154 


2213 


Total 


8800 


1633 


2754 


3557 


1984 


18728 



Table 15 

Category X Teachers and Category X Syntax 
Tests of Independence 





Category 


Info 


df 


Signif. 




*1 


9.858 


9 


.36 




2, 


4.904 


9 


.84 


Teachers 




4.904 




.84 


2 4 


9 




All others . 


0.0 


9 


1.00 




X 1 


221.501 


4 


.001 




l 2 


105.376 




.001 




*3 


2.866 




.58 


Syntax 


2 4 


4.412 




.35 




2 5 


4.799 




.31 




2 1> V 2 3 


0.00 




1.00 
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Table 17 



Ttoo-Way Tests of 


Independence of Parts of 


0S2 




Component 


Info. 


df 


P 


1 1 t0 2 5 + 3 10 + 5 10 X 3 10 


4 * 4 4 > 4 5 

*10 *20 20 50.39 


9 


0.001 


h to 2 5 X 5 10 


0.0 


9 


1.00 


3 10 + 4 10 + 4 20 X 5 20 


0.0 


9 


1.00 


1 1* 2 + 13 X 2 1, 2, 3, 4 + 


15 0.0 


9 


1.00 



1 1, 2 + 13 X 2 1, 



Table 18 

Two-Way Tests of Independence for Syntax X Teachers in 0S2 





Syntax 


Info 


df 


P 




Level 1 


124.111 


9 


0.001 




Level 2 


34.303 


9 


0.001 


Teachers 


Level 3 


176.415 


9 


0.001 




Level 4 


111.328 


9 


0.001 




Level 5 


65.600 


9 


0.001 
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Table .19 

Geometric Mean Scores 



Teacher 


MID 


MIDI 


1 M. N. 


41.45 


16.13 


2 C. T. 


39.35 


13.84 


3 G. G. 


39.57 


14.99 


4 S. S. 


34.78 


. 11.15 


5 J. A. 


40.51 


16.13 


6 P. B. 


43.52 


16.54 


7 R. S. 


40.39 


16.48 


8 A. D. 


47. 73 


14.91 


9 F. W. 


42.82 


17.19 


10 D. W. 


42.15 


16.16 
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Table 20 

Meant Ratings for 17 Questions of Student 
Questionnaire and Grand Mean 




ERIC 
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Table 21 

Evaluation Variables 



Teachers 


1 


2 


3 


4 5 

Joint Observers' Ratines 


WMID 


MIDI 


Ss Ratings 


Command 
of French 


Teaching 

Techniques 


1 M. N. 


5 


5.5 


6 


3 


3 


2 C. T. 


9 


9 


7 


8 


5 


3 6* 6* 


8 


8 


8.5 


5 


10 


4 S. S. 


10 


10 


5 


6 


6 


5 J. A. 


6 


5.5 


8.5 


9 


4 


6 F. B. 


1 


2 


4 


7 


9 


7 R. S. 


7 


3 


10 


10 


8 


8 A. D. 


4 


8 


3 


2 


2 


9 F. W. 


2 


1 


1 


4 


7 


10 D. W. 


3 


4 


2 


1 


1 



Table 22 

Spearman Eho Correlations for 









Evaluation 


Criteria 






Pairs 


2 

£ D (Squared Difference) 








1, 


2 


46.5 


** 








1, 


3 


57.5 


** 


2 

Significance for D 


Direction 


1, 


4 


96 


* 


*.05 


99.3 




1, 


5 


140 




**.025 


59.4 


Positive 


2, 


3 


122.5 




***.005 


39.6 




2, 


4 


167.5 




*.05 


260.7 




2, 


5 


204.5 




**.025 


270.6 


Negative 


3. 


4 


43.5 


** 
















***.005 


290.4 




3# 


5 


103.5 










a, 


5 


76 


* 
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Appendix 1 

Paper Tape Punch and Data Multiplexor 

The data recording system consists of a 50 character-per-second 
paper tape punch (able to punch standard 8 level paper tape) and 
associated electronics necessary to gather button press data from up 
to three stations plus data In the form of a binary number from an 
Interval timer. 



Box 1 



Box 2 



Box 3 



Timer 



Each recording station consists of a keyboard of up to 16 buttons, i 
Whenever an observer pushes a button, the 4-bit code for that button plus 
a 2-bit station-identifying code is gated to the data multiplexor. The 
entire code is then either punched on the paper tape, or held until the 
punch is free, and is then punched. Therefore, data from any station 
are recorded as soon as possible. If the multiplexor receives a special 
"terminating code" from any station, it punches this code and then 
immediately punches the contents of the timer. Thus, when an observer 
completes an n-tally sequence the time since the last completion can be 
recorded. 
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Appendix 2 

0S1 One-Step Category Tables for Teachers 
0S1 One-Step Category Table for Teacher No. 1 





1 


2 


3 


4 


5 


6 


7 


8 


9 


10 


11 


12 


13 


14 


15 


16 


17 


18 


1. 


3 


0 


1 


0 


0 


0 


0 


0 


5 


1 


0 


0 


0 


0 


0 


0 


0 


0 


2. 


1 


11 


0 


2 


0 


0 


4 


2 


14 


7 


2 


0 


0 


0 


32 


17 


8 


10 


3. 


2 


1 


1 


43 


7 


16 


9 


8 


28 


28 


2 


1 


2 


0 


5 


0 


0 


9- 


4. 


0 


1 


40 


11 


4 


5 


3 


7 


33 


21 


0 


0 


18 


25 


3 


3 


0 


3 


5. 


0 


2 


0 


0 


71 


0 


2 


4 


13 


35 


0 


0 


1 


209 


2 


0 


0 


3 


6 . 


0 


0 


0 


0 


0 


9 


0 


2 


1 


3 


1 


0 


0 


17 


43 


3 


0 


16 


7. 


0 


3 


0 


0 


1 


0 


99 


2 


6 


1 


2 


0 


0 


0 


29 


5 


1 


60 


8. 


0 


4 


0 


0 


1 


1 


0 


42 


5 


0 


3 


0 


0 


0 


44 


1 


0 


29 


9. 


1 


33 


4 


9 


56 


11 


24 


13 


647 


39 


0 


4 


3 


0 


9 


5 


9 


42 


10. 


0 


4 


0 


26 


49 


4 


4 


2 


13 


88 


5 


2 


10 


33 


19 


5 


0 


25 


11. 


0 


1 


1 


3 


0 


1 


2 


2 


8 


7 


8 


0 


11 


7 


14 


3 


1 


6 


12. 


1 


3 


1 


4 


2 


0 


3 


3 


11 


5 


4 


14 


0 


0 


1 


0 


0 


1 


13. 


0 


1 


16 


13 


0 


1 


0 


0 


0 


7 


8 


1 


66 


0 


0 


1 


0 


1 


14. 


“I 


4 


19 


9 


145 


31 


2 


14 


37 


18 


6 


3 


0 


17 


0 


0 


0 


4 


15. 


1 


27 


75 


54 


0 


2 


2 


11 


13 


5 


33 


27 


0 


0 


127 


1 


0 


32 


16. 


0 


4 


1 


2 


0 


4 


15 


0 


29 


2 


0 


0 


0 


0 


0 


6 


0 


3 


17. 


0 


0 


1 


0 


0 


1 


0 


0 


17 


0 


0 


0 


0 


0 


0 


0 


6. 


1 


18. 


0 


'll 


2 


0 


6 


9 


40 


18 


29 


21 


1 


1 


4 


2 


82 


16 


1 


109 



Preceding Category Versus Following Category 
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Appendix 2 - continued 

0S1 One-Step Category Table for Teacher No. 2 





1 


2 


3 


4 


5 


6 


7 


8 


9 


10 


11 


12 


13 


14 


15 


16 


17 


18 


1. 


2 


0 


0 


0 


0 


0 


3 


0 


4 


4 


1 


0 


0 


0 


0 


0 


1 


2 


2. 


0 


10 


2 


0 


0 


0 


3 


0 


2 


6 


0 


0 


0 


2 


47 


3 


6 


7 


3. 


2 


0 


6 


90 


0 


8 


14 


6 


23 


22 


2 


1 


1 


1 


2 


2 


0 


8 


4. 


0 


0 


50 


52 


4 


24 


18 


14 


95 


57 


3 


1 


5 


7 


8 


1 


2 


17 


5. 


0 


0 


0 


0 


34 


6 


0 


1 


5 


27 


0 


0 


5 


37 


0 


0 


0 


1 


6. 


1 


0 


0 


0 


0 


34 


1 


5 


4 


2 


1 


0 


0 


20 


54 


0 


0 


32 


7. 


0 


10 


1 


1 


0 


0 


96 


1 


17 


6 


0 


0 


0 


0 


38 


3 


1 


54 


8. 


1 


2 


0 


2 


0 


2 


1 


21 


15 


3 


0 


0 


0 


1 


50 


0 


0 


40 


9. 


3 


9 


8 


27 


20 


10 


44 


41 


834 


72 


6 


1 


4 


0 


16 


3 


10 


55 


10. 


4 


4 


1 


8 


27 


16 


15 


6 


33 


242 


19 


1 


46 


57 


25 


4 


1 


59 


11. 


0 


0 


2 


3 


0 


0 


0 


4 


12 


20 


13 


0 


30 


16 


23 


0 


0 


11 


12. 


0 


1 


0 


0 


0 


0 


1 


2 


6 


4 


4 


5 


0 


0 


5 


0 


0 


3 


13. 


1 


0 


20 


14 


3 


6 


1 


2 


2 


15 


32 


2 


142 


0 


1 


0 


0 


6 


14. 


0 


2 


6 


38 


27 


14 


0 


1 


8 


28 


13 


0 


3 


62 


1 


0 


0 


2 


15. 


1 


13 


88 


116 


0 


21 


3 


6 


6 


13 


29 


15 


0 


0 


123 


2 


0 


54 


16. 


0 


3 


1 


0 


0 


2 


2 


1 


18 


5 


0 


0 


1 


0 


1 


2 


0 


0 


17. 


0 


3 


1 


0 


0 


0 


0 


0 


17 


1 


0 


0 


.0 


0 


0 


0 


10 


2 


18. 


2 


31 


2 


7 


1 


11 


26 


27 


61 


41 


11 


5 


10 


2 


96 


16 


3 


90 
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Appendix 2 - continued 



0S1 One-Step Category Table for Teacher No. 3 



1 


2 


3 


4 


5 


6 


7 


8 


9 


10 


11 


12 


13 


14 


15 


16 


17 


1? 


1. 4 


1 


0 


5 


1 


2 


3 


0 


5 


0 


0 


0 


0 


0 


0 


3 


i 


0 


2. 0 


1 


1 


1 


0 


0 


0 


1 


1 


3 


1 


0 


0 


0 


32 


2 


8 


0 


3. 2 


0 


0 


69 


3 


12 


5 


2 


27 


17 


1 


0 


1 


0 


4 


0 


0 


3 


4. 3 


2 


29 


31 


9 


115 


30 


18 


114 


36 


1 


0 


3 


3 


2 


2 


1 


14 


5. 0 


0 


0 


0 


52 


3 


5 


14 


35 


44 


0 


0 


0 


57 


2 


0 


0 


6 


6. 0 


1 


0 


4 


1 


34 


1 


4 


13 


29 


1 


0 


0 


45 


111 


0 


0 


23 


7. 1 


9 


0 


0 


3 


1 


165 


1 


24 


4 


1 


0 


0 


2 


63 


6 


0 


66 


8. 2 


10 


0 


1 


2 


1 


3 


47 


19 


9 


2 


0 


0 


1 


54 


1 


1 


30 


9. 5 


6 


3 


41 


83 


28 


57 


61 


'368 


61 


3 


0 


3 


0 


24 


11 


12 


51 


10. 0 


5 


1 


10 


36 


30 


11 


6 


30 


135 


4 


0 


10 


54 


18 


2 


0 


33 


11. 0 


1 


0 


1 


0 


1 


1 


2 


17 


0 


3 


0 


13 


2 


71 


1 


1 


4 


12. 0 


0 


0 


1 


0 


1 


2 


4 


3 


2 


1 


0 


0 


0 


4 


0 


0 


1 


13. 0 


0 


4 


6 


0 


0 


2 


0 


3 


2 


13 


0 


49 


0 


0 


0 


0 


1 


14. 1 


0 


27 


48 


23 


12 


1 


4 


40 


8 


2 


0 


0 


13 


0 


0 


0 


0 


15. 1 


8 


81 


190 


1 


14 


13 


4 


18 


7 


79 


19 


0 


0 


98 


2 


0 


13 


16. 4 


0 


0 


1 


0 


2 


5 


1 


19 


1 


1 


0 


0 


0 


1 


2 


0 


2 


17. 0 


2 


0 


0 


0 


2 


0 


0 


18 


0 


0 


0 


0 


0 


1 


0 


5 


2 


18. 2 


5 


0 


4 


4 


9 


42 


14 


63 


26 


5 


0 


1 


2 


63 


7 


1 


35 
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0S1 One-Step Category Table for Teacher No. 4 





1 


2 


3 


4 


5 


6 


7 


8 


9 


10 


11 


.12 


13 


14 


15 


16 


17 


18 


1. 


5 


1 


3 


0 


3 


0 


9 


0 


11 


7 


O' 


0 


0 


0 


2 


4 


0 


3 


2. 


6 


8 


0 


2 


0 


2 


6 


0 


7 


15 


1 


1 


0 


6 


18 


11 


2 


4 


3. 


6 


4 


1 


81 


26 


21 


39 


4 


26 


24 


1 


0 


12 


“1 


3 


G 


0 


0 


4. 


5 


3 


50 


10 


3 


54 


33 


1 


27 


40 


2 


0 


10 


31 


7 


0 


0 


2 


5. 


2 


3 


0 


0 


175 


7 


6 


2 


2 


32 


0 


0 


0 


271 


0 


0 


0 


8 


6. 


0 


0 


0 


0 


0 


3 


0 


0 


0 


3 


0 


0 


0 


71 


62 


5 


0 


22 


7. 


5 


1 


0 


3 


1 


0 


139 


0 


23 


3 


1 


0 


, 0 


0 


80 


8 


0 


97 


8. 


0 


1 


0 


0 


0 


0 


1 


10 


3 


0 


1 


0 


0 


0 


10 


1 


0 


8 


9. 


7 


22 


9 


5 


18 


5 


42 


4 


370 


32 


2 


0 


1 


1 


3 


11 


2 


35 


10. 


2 


2 


0 


9 


46 


1 


3 


3 


17 


83 


3 


0 


16 


68 


9 


1 


0 


36 


li. 


0 


2 


4 


1 


0 


0 


4 


0 


3 


4 


3 


0 


89 


1 


61 


1 


1 


3 


12. 


0 


0 


0 


0 


0 


1 


4 


1 


0 


0 


1 


0 


0 


0 


0 


0 


1 


0 


13. 


2 


6 


35 


7 


0 


0 


2 


1 


1 


4 


76 


0 


128 


0 


1 


0 


0 


1 


• 

i— I 


2 


2 


60 


61 


228 


55 


1 


1 


14 


22 


1 


0 


0 


117 


0 


0 


0 


0 


15. 


3 


14 


86 


95 


0 


2 


8 


0 


7 


2 


76 


6 


0 


0 


74 


0 


0 


17 


16. 


1 


7 


1 


2 


0 


6 


16 


0 


17 


2 


3 


0 


0 


0 


0 


19 


0 


4 


17. 


0 


1 


0 


0 


0 


0 


0 


0 


3 


0 


1 


1 


0 


0 


0 


0 


1 


0 


00 

• 


2 


11 


0 


2 


8 


9 


48 


8 


38 


25 


5 


1 


2 


v<3 


60 


17 


0 


73 
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0S1 One-Step Category Table for Teacher No. 5 





1 


2 


3 


4 


5 


6 


7 


8 


9 


10 


11 


12 


13 


14 


15 


16 


17 


18 


1. 


3 


0 


1 


o 


2 


1 


0 


0 


6 


0 


0 


0 


0 


0 


0 


1 


0 


9 


2. 


1 


4 


0 


1 


0 


0 


2 


0 


7 


4 


0 


0 


1 


1 


7 


5 


10 


12 


3. 


1 


3 


1 


37 


20 


10 


3 


0 


10 


15 


1 


0 


0 


0 


3 


0 


0 


8 


4. 


3 


0 


23 


28 


1 


2 


8 


2 


22 


10 


0 


0 


36 


189 


8 


1 


0 


7 


5. 


1 


1 


1 


0 


63 


6 


4 


1 


7 


11 


0 


0 


91 


311 


2 


0 


0 


5 


6. 


1 


0 


0 


1 


0 


118 


0 


2 


3 


2 


1 


0 


0 


8 


110 


0 


1 


30 


7. 


0 


1 


0 


2 


0 


0 


44 


-L 


8 


0 


0 


0 


0 


1 


21 


2 


0 


10 


8. 


0 


0 


0 


1 


1 


0 


2 


14 


11 


1 


0 


0 


0 


0 


20 


0 


1 


18 


9. 


6 


25 


4 


16 


21 


14 


15 


18 


958 


22 


2 


0 


0 


0 


6 


8 


13 


53 


10. 


2 


2 


0 


3 


32 


5 


1 


3 


16 


84 


0 


1 


10 


20 


6 


0 


0 


30 


11. 


0 


0 


0 


6 


1 


1 


0 


3 


2 


0 


10 


0 


18 


3 


70 


2 


0 


9 


12. 


0 


0 


0 


0 


1 


3 


0 


5 


5 


2 


10 


1 


0 


0 


2 


0 


0 


0 


13. 


0 


0 


15 


31 


84 


0 


0 


3 


2 


1 


18 


1 


40 


0 


0 


1 


0 


4 


14. 


5 


2 


44 


66 


269 


93 


3 


4 


16 


15 


3 


2 


2 


79 


0 


0 


0 


15 


15. 


0 


6 


21 


140 


0 


9 


1 


8 


11 


3 


67 


24 


0 


0 


105 


2 


0 


20 


16. 


1 


2 


0 


1 


0 


3 


0 


1 


14 


1 


6 


0 


0 


0 


1 


4 


0 


2 


17. 


0 


2 


0 


0 


0 


0 


0 


0 


24 


0 


0 


0 


0 


0 


0 


0 


30 


2 


18. 


1 


7 


2 


5 


9 


12 


7 


4 


58 


44 


7 


0 


2 


6 


56 


10 


3 


40 
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0S1 One-Step Category Table for Teacher No. 6 





1 


2 


3 


4 


5 


6 


7 


8 


9 


10 


11 


12 


13 


14 


15 


16 


17 


1. 


1 


1 


1 


0 


0 


0 


1 


0 


6 


1 


0 


0 


0 


0 


1 


1 


0 


2. 


1 


4 


0 


0 


0 


1 


3 


0 


3 


3 


0 


0 


1 


0 


33 


7 


16 


3. 


2 


4 


9 


56 


12 


33 


17 


17 


54 


49 


2 


0 


13 


0 


6 


1 


2 


4. 


1 


2 


69 


5 


0 


5 


8 


8 


20 


20 


2 


0 


3 


3 


4 


0 


0 


5. 


0 


0 


0 


0 


34 


2 


2 


4 


2 


10 


0 


0 


0 


59 


0 


0 


1 


6. 


0 


0 


0 


0 


0 


24 


2 


2 


1 


17 


4 


0 


0 


53 


40 


3 


0 


7. 


0 


8 


0 


0 


1 


1 


78 


1 


29 


6 


0 


0 


0 


0 


39 


6 


1 


8. 


1 


7 


1 


0 


0 


1 


1 


50 


6 


1 


1 


0 


0 


1 


43 


2 


X 


9. 


2 


12 


9 


4 


15 


14 


54 


22 


602 


36 


0 


0 


4 


0 


8 


15 


22 


10. 


0 


3 


3 


0 


14 


22 


12 


8 


16 


134 


3 


0 


13 


63 


18 


3 


2 


11. 


0 


1 


2 


1 


0 


1 


0 


1 


2 


2 


0 


0 


60 


8 


35 


1 


0 


12. 


0 


0 


C 


0 


0 


2 


1 


1 


2 


0 


0 


0 


0 


0 


0 


0 


0 


13. 


0 


0 


30 


3 


0 


0 


0 


0 


2 


2 


60 


0 


206 


0 


0 


0 


0 


14. 


0 


1 


49 


17 


36 


31 


1 


11 


8 


13 


8 


2 


0 


40 


1 


0 


1 


15. 


1 


14 


113 


75 


0 


5 


4 


10 


6 


8 


32 


3 


0 


0 


93 


3 


2 


16. 


2 


3 


0 


0 


1 


6 


6 


2 


24 


10 


3 


0 


0 


0 


1 


4 


0 


17. 


0 


2 


0 


0 


0 


0 


0 


2 


39 


3 


0 


0 


0 


0 


0 


0 


30 


18. 


4 


22 


10 


2 


7 


15 


49 


20 


80 


40 


1 


1 


3 


4 


61 


19 


2 



Preceding category versus following category 



0 




18 

2 

12 

19 

13 

6 

17 

70 

43 

83 

41 

2 

0 

0 

12 

14 

3 

4 

95 
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0S1 One-Step Category Table for Teacher No. 7 





1 


2 


3 


4 


5 


6 


7 


8 


9 


10 


11 


12 


13 


14 


15 


16 


17 


18 


1 


0 


0 


0 


0 


0 


0 


4 


0 


2 


0 


0 


0 


0 


0 


1 


0 


0 


6 


2. 


2 


1 


0 


1 


0 


0 


1 


0 


3 


5 


0 


0 


0 


0 


3 


0 


2 


5 


3. 


0 


2 


0 


38 


2 


8 


6 


3 


15 


5 


2 


0 


0 


0 


3 


0 


0 


5 


4. 


2 


4 


27 


9 


2 


63 


49 


3 


27 


37 


1 


0 


0 


0 


5 


0 


0 


5 


5. 


0 


0 


0 


0 


78 


0 


6 


14 


6 


23 


1 


0 


0 


99 


1 


0 


0 


3 


6. 


0 


0 


0 


1 


1 


33 


1 


0 


3 


1 


0 


0 


0 


43 


61 


5 


0 


14 


7. 


2 


2 


0 


0 


1 


1 


26C 


5 


48 


1 


1 


1 


,0 


0 


107 


19 


1 


199 


8. 


0 


0 


0 


1 


4 


0 


1 


82 


31 


0 


1 


0 


0 


0 


29 


1 


0 


71 


9. 


1 


7 


0 


3 


30 


3 


97 


36 


4^6 


47 


1 


0 


0 


0 


11 


1 


3 


98 


10. 


0 


0 


0 


1 


35 


14 


12 


12 


26 


75 


7 


0 


24 


19 


6 


0 


0 


56 


• 

i— 1 

iH 


0 


0 


1 


0 


2 


1 


5 


1 


8 


16 


3 


0 


15 


0 


12 


0 


0 


3 


12. 


0 


0 


0 


2 


0 


1 


5 


2 


8 


2 


2 


0 


0 


0 


4 


0 


0 


1 


13. 


0 


0 


10 


5 


1 


1 


1 


1 


0 


1 


21 


0 


90 


0 


3 


0 


0 


1 


14. 


0 


1 


0 


40 


66 


17 


2 


5 


18 


12 


0 


0 


0 


29 


0 


0 


0 


1 


15. 


5 


1 


48 


131 


0 


9 


29 


6 


15 


4 


25 


24 


0 


1 


136 


0 


0 


26 


16. 


0 


1 


0 


0 


0 


5 


19 


0 


6 


1 


1 


0 


0 


0 


0 


1 


0 


2 


17. 


0 


0 


0 


0 


0 


0 


0 


0 


6 


1 


0 


0 


0 


0 


0 


0 


9 


0 


• 

00 
I— 1 


1 


4 


3 


2 


9 


7 


150 


51 


116 


55 


2 


2 


5 


0 


78 


9 


1 


126 



Preceding category versus following category 



ERIC 



Capelle, Jarvella, & Revelle 
Appendix 2 - continued 



59 



0S1 One-Step Category Table for Teacher No. 8 





1 


2 


3 


4 


5 


6 


7 


8 


9 


10 


11 


12 


13 


14 


15 


16 


17 


18 


1. 


2 


1 


0 


0 


0 


0 


0 


0 


2 


5 


0 


0 


1 


0 


1 


0 


0 


0 


2. 


J. 


27 


0 


1 


0 


1 


1 


1 


14 


14 


0 


0 


1 


0 


31 


8 


50 


12 


3. 


0 


12 


6 


72 


16 


30 


13 


26 


38 


34 


7 


0 


6 


0 


10 


0 


0 


17 


4. 


2 


8 


123 


37 


7 


17 


10 


18 


75 


27 


4 


0 


7 


7 


6 


C 


3 


7 


5. 


0 


3 


2 


0 


45 


4 


3 


12 


10 


31 


4 


0 


0 


160 


18 


0 


0 


10 


6. 


0 


0 


0 


1 


1 


44 


0 


2 


2 


2 


3 


0 


0 


42 


60 


3 


1 


26 


7. 


4 


8 


1 


1 


3 


0 


91 


3 


14 


2 


0 


0 


0 


0 


48 


3 


2 


19 


8. 


0 


12 


0 


0 


1 


0 


2 


79 


19 


2 


2 


0 


0 


1 


106 


2 


3 


28 


9. 


1 


52 


11 


15 


27 


9 


30 


62 


479 


42 


5 


1 


1 


0 


15 


15 


34 


46 


10. 


1 


6 


0 


13 


44 


37 


15 


10 


27 


89 


23 


0 


17 


44 


10 


2 


1 


27 


11. 


0 


1 


4 


1 


2 


1 


0 


5 


5 


4 


A 


0 


57 


4 


73 


1 


2 


5 


12. 


0 


0 


0 


0 


3 


0 


1 


8 


5 


3 


4 


0 


1 


0 


5 


0 


0 


1 


13. 


0 


1 


12 


21 


0 


0 


0 


0 


1 


13 


40 


3 


89 


0 


0 


0 


0 


1 


14. 


0 


0 


32 


10 


137 


27 


0 


2 


5 


40 


2 


0 


0 


13 


0 


0 


0 


3 


15. 


0 


9 


91 


178 


6 


1 


6 


13 


13 


15 


59 


25 


0 


0 


120 


1 


1 


23 


16. 


0 


0 


2 


0 


1 


1 


3 


1 


27 


4 


1 


0 


0 


0 


1 


12 


0 


1 


17. 


0 


7 


1 


5 


1 


1 


1 


1 


75 


3 


1 


0 


0 


0 


0 


0 


71 


8 


18. 


1 


15 


2 


3 


8 


14 


23 


14 


34 


35 


10 


2 


1 


'•.0 


57 


7 


7 


66 
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0S1 One-Step Category Table for Teacher No. 9 



1 


2 


3 


4 


5 


6 


7 


8 


9 


10 


11 


12 


13 


14 


15 


16 


17 


18 


1. 3 


0 


1 


0 


0 


1 . 


2 


0 


2 


1 


0 


0 


0 


0 


0 


0 


1 


2 


2. 0 


4 


0 


0 


0 


2 


2 


1 


2 


2 


1 


0 


1 


0 


32 


5 


8 


12 


3. 3 


1 


7 


15 


1 


35 


13 


0 


16 


58 


2 


0 


5 


0 


3 


0 


1 


37 


4. 0 


0 


93 


25 


0 


19 


10 


1 


26 


26 


1 


0 


5 


7 


10 


0 


2 


23 


5. 0 


0 


0 


0 


100 


9 


0 


0 


5 


19 


0 


0 


1 


108 


3 


0 


0 


2 


6. 0 


2 


0 


0 


0 


82 


0 


2 


1 


12 


3 


0 


4 


20 


87 


3 


0 


40 


7. 1 


2 


0 


0 


0 


0 


136 


2 


3 


1 


4 


0 


1 


0 


53 


4 


1 


43 


8. 1 


1 


0 


0 


0 


0 


0 


30 


3 


1 


0 


1 


0 


0 


22 


2 


0 


14 


9. 1 


9 


10 


7 


18 


11 


13 


11 


525 


33 


11 


1 


0 


0 


16 


8 


18 


71 


10. 1 


3 


0 


8 


42 


27 


10 


4 


16 


140 


3 


0 


42 


18 


23 


3 


1 


50 


11. 0 


3 


1 


1 


1 


4 


4 


0 


16 


6 


17 


0 


63 


3 


156 


6 


2 


10 


12. 0 


2 


0 


0 


0 


3 


1 


2 


7 


1 


13 


0 


0 


0 


3 


1 


0 


1 


13. 0 


1 


23 


27 


0 


2 


1 


0 


2 


5 


58 


1 


62 


0 


0 


4 


1 


3 


14. 0 


1 


19 


6 


80 


15 


0 


3 


15 


12 


1 


0 


0 


15 


0 


0 


0 


4 


15. 2 


14 


37 


154 


1 


13 


4 


4 


13 


6 


160 


28 


0 


0 


104 


2 


0 


29 


16. 1 


4 


1 


1 


1 


5 


4 


0 


20 


3 


4 


0 


1 


0 


4 


2 


0 


3 


17. 0 


2 


0 


1 


0 


0 


0 


0 


31 


0 


0 


0 


0 


0 


0 


0 


18 


2 


18. 0 


23 


5 


3 


3 


28 


51 


15 


61 


64 


15 


3 


5 


0 


55 


14 


1 


86 



Preceding category versus following category 
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0S1 One-Step Category Table for Teacher No. 10 





1 


2 


3 


4 


5 


6 


7 


8 


9 


10 


11 


12 


13 


14 


15 


16 


17 


18 


1 . 


7 


0 


0 


0 


1 


1 


5 


1 


7 


6 


0 


0 


0 


0 


3 


4 


0 


4 


2. 


2 


15 


2 


2 


0 


2 


7 


3 


7 


7 


0 


0 


2 


0 


40 


8 


3 


11 


3. 


2 


2 


4 


83 


2 


10 


19 


9 


22 


23 


4 


1 


2 


0 


7 


0 


0 


5 


4. 


1 


7 


69 


53 


1 


83 


71 


30 


62 


28 


2 


3 


13 


11 


12 


7 


0 


16 


5. 


0 


0 


0 


0 


10 


0 


0 


1 


1 


17 


0 


0 


0 


132 


2 


0 


0 


1 


6. 


0 


4 


0 


0 


0 


28 


0 


1 


4 


2 


1 


0 


1 


19 


77 


4 


0 


40 


7. 


1 


13 


0 


2 


0 


0 


185 


3 


11 


3 


4 


1 


0 


0 


101 


20 


0 


121 


8. 


1 


3 


0 


1 


0 


1 


1 


117 


18 


7 


2 


0 


0 


0 


95 


6 


0 


51 


9. 


8 


16 


1 


11 


6 


6 


43 


49 


414 


44 


7 


1 


5 


0 


24 


6 


14 


42 


10. 


3 


2 


0 


9 


34 


9 


14 


12 


17 


112 


0 


1 


24 


25 


30 


8 


2 


42 


11. 


0 


0 


1 


5 


0 


2 


9 


3 


11 


6 


9 


0 


43 


6 


90 


6 


0 


11 


12. 
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3 


0 
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1 


0 


3 


3 


5 


2 


1 


1 
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0 


3 


0 


0 


2 
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0 


2 
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17 


0 


0 


0 


0 


4 


5 


43 


0 


94 


0 


0 


1 


0 


1 


14. 


0 


1 


18 


20 
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9 


0 


11 


7 


14 


5 


0 


0 


27 


0 


0 


0 


2 


15. 


6 


12 


81 


258 


0 


11 


19 


14 


14 


14 


111 


16 


0 


0 


193 


8 


1 


50 


16. 


4 


6 
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5 


0 


9 


17 


2 


23 


16 
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0 


0 


3 


7 


0 


9 


17. 


0 


1 


0 


0 


0 
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1 


1 


19 


0 


0 
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0 


0 


0 
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12 


1 
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• 


4 


24 


1 


3 


2 


10 


71 


43 


51 


37 
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22 


3 
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Fevrier 1970 
n ombre de points 
note 



Appendix 3 
Mid-Term Exam 
(Examen de Ml-Semestre) 

Franqais 102 

nom 

section 

professeur 



I. Comprehension Orale 



A. Vous allez entendre six questions * Chaque question est 
suivie d'une reponse. Si la reponse donnee vous semble 
possible ou logique, mettez un c?r.:le autour de VRAI. 
Sin on, mettez un cercle autour de FAUX. (6 points) 



1. 


VRAI 


FAUX 


4. 


VRAI 


FAUX 


2. 


VRAI 


FAUX 


5. 


VRAI 


FAUX 


3. 


VRAI 


FAUX 


6. 


VRAI 


FAUX 



B. Vous allez entendre six phrases. Chaque phrase contient 
un nombre. Ecrivez le nombre que vous entendez dans 
chaque phrase. (6 points) 



1 . 



4 . 



2 . 



5. 



3. 



6 . 



C. Mettez les phrases suivants au passe compose. (6 points) 

1. 4. 

2. 5. 



3. 



6 . 



O 
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D. Repondez aux questions suivantes, af firmativement ou negativement 
selon le cas, en employ ant des pronoms appropries. N ' employez 
pas la forme nous ni la forme tu dans vos reponses. (6 points) 

1. Oui, 

2. Non, 

3. Si, 

E. Repondez aux questions suivantes par des phrases completes 
(c'est-a-dire, sujet-verbe-complement) . N* employ ez pas la 
forme nous ni la forme tu dans vos reponses. (6 points) 



1 . 



2 . 



3. 



II. Partie Ecrite 



A. Posez une question sur la partie soulignee de chaque phrase, 
d'apres ce modele: le livre est rouge. Ou'est-ce qui est 
rouge ? (10 points) 

1. II parlait de la mere de Jacques . 

2. Je prends un verre de lait . 

3. Cette rue mene a la gare . 

4. M. Blanc m'a donne l'adresse. 

5 . Je connais Yvonne depuis six mots . 



B. Completez les phrases suivantes en mettant un ou deux mots 
logiques et appropries dans chaque parenthese. (8 points) 



1 . 

2 . 

3. 

4. 

5. 

6 . 

7. 

8 . 



La regie ( ) vous oibeissez est necessaire. 

M. Dubois est le professeur ( ) j'ai lu la these. 

II est douteux que nous ( ) contents. 

EBt-il vrai que tu ( ) cela? 

Ce film est agreable ( ) voir. 

Est-il difficile ( ) monter l'escalier? 

J'aime la robe ( ) vous portez aujourd'hui. 

J'ai ( ) dit cela. 
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C. Completes lcs phrases suivantes. Soyez aussi bref que possible. 
Faites attention au temps du verbe que vous employez. (5 points) 

1. Si je travaille, je 

2. Nous etions deja partis pour Paris quand vous 

3. Telephone-moi quand tu 

: 4. II est temps que nous 

5. Je jouais du piano pendant que Jeanne 

III. Lecture 



A. Lisez rapidement les passages suivants et puis indiquez si les 
commentaires au sujet de ces passages sont VRAIS ou FAUX en 
mettant un cercle autour de V. ou F. (9 points) 

Peut-Atre que l’eglise et son enchevAtrement de styles ont 
donne le ton au quartier. Ici voisinent l 1 erudition et les 
plaisirs, la futilite et la philosophic. Ces contrastes 
existent dans tout le quartier. Tout autour de nous, tu 
remarques la vie agitee et bruyante d* artistes, d'etudiants, 
de bohemes; mais a deux pas d'ici tu trouveras le grand calme 
de la Place de Furstenberg ou Delacroix installa son atelier 
et ou il mourut. 

1. V. F. Ce passage parle de l 1 atmosphere variee du quartier 

2. V. F. L 1 expression a deux pas veut dire la mAme chose que 

a peu pres . 

3. V. F. L'atelier de Delacroix etait l'endroit ou il travaillait. 



Les cinemas du Quartier Latin semblent defier la grande publicite, car 
ils offrent chaque semaine de bons films qui n'ont pas requ un accueil 
favorable de la critique ou du grand public. Quand un film de moins 
bonne qualite vient interrompre la serie, le spectacle passe parfois 
dans la salle. Les etudiants manifestent leur mecontentement par des 
reflexiens drdles , ou bien la salle toute entiere participe a l f action 
du film et reagit en mAme temps que les acteurs. C'est ainsi que le 
cinema du Quartier Latin reste une distraction amusante quelle que 
soit la qualite du film projete. 



4. V. F. 

5. V. F. 

6. V. F. 



D'apres le passage, le grand public et les spectateurs 
des cinemas du Quartier Latin aiment la m&me sorte de films 
Quelquefois est un synonyme de l'adverbe parfois qui se 
trouve a la ligne 6. 

Les cinemas du Quartier Latin manquent d f inter At quand un 
film de moins bonne qualite est projete. 



t 
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Peter avalt mis quelques jours avant de s' adapter au rythme des repas 
franfais. Un dejeuner au restaurant prend souvent d ' une heure et demie 
a deux heures. Le diner dure davantage et ne commence que vers huit 
heures et mAme souvent plus tard. Cecl le changealt naturellement de 
ses habitudes americaines. J' avals tenu a lul presenter le c6te tres 
special de la vie franqaise qu'est la cuisine. En France on dit "l' Art 
culinaire". Peter n' avalt pas compris. 

7. V. F. Selon le passage, le dejeuner dans un restaurant 

en France commence a une heure et dure jusqu’a deux 
heures . 

8. V. F. Le diner dure plus longtemps que le dejeuner. 

9. V. F. L' express! on j'avais tenu a, veut dire I'avais 

insiste pour (expression qui se trouve a la 
ligne 5-7) . 

B; Lisez les passages suivants et puis repondez en franfais aux 

questions qui les accompagnent. Repondez aussi brievement 

que possible. Vous n’etes pas oblige de repondre par des 

phrases completes. (8 points) 

Quel charme, en toute saison, a aussi le Jardin du Luxembourg: Le 

Palais, que loge le Senat, presente un exemple parfait du style du 
Roi Soleil, mais en beaucoup plus intime que Versailles. Autour des 
bassins ou evoluent des flottilles, guidees par une multitude 
d'enfants, la vie est intense. C'est surtout la 1' attraction de ce 
jardin avec son the At re de Marionnettes ou, parmi les enfants, on 
peut trouver de nombreux spectateurs adultes, fervents de ce genre 
de spectacle. La proximlte de la Sorbonne et du Quartier Latin permet 
a de nombreux etudiants de venir se donner rendez-vous dans ce beau 
jardin et d'y discuter leurs problemes particuliers ou academiques. 

1. Qu'est-ce qui compose des "flottilles" (a la ligne 5)? 

2. Donnez deux exemples du style architectural de Louis XIV. 

3. De quel spectacle les adultes sont-ils fervents? 

4. Donnez un synonyme de l'adjectif particuliers qui se 
trouve dans la derniere phrase du passage. 

5. Pourquoi le Jardin du Luxembourg est-il en particulier un 
lieu favor! de rencontre parmi les etudiants? 

La Seine est l'Atre essential dans la vie de Paris. Elle forme un grand 
arc qui part du sud-est, monte vers le nord, puis redescend vers le sud- 
ouest. Cet arc divise la ville en deux parties inegales . La rive droite 
est la plus importante. Les rois de France sont venue s'y installer des 
le XIV 6 siecle. Ils ont ete suivis par les nobles, les bourgeois riches, 
les commerqants. Cette partie de Paris est devenue la capitale de 
1' elegance, du the At re , des plaisirs. La rive gauche a et^ et est 
encore la region du monde intellectuel, judiciaire et eccl^siastique 

6. Ou est-ce que les rois de France se sont installs a 
partir du XIV 6 siecle? 

7. Comment s'appellent les deux bords de la Seine? 

8. A quo! se rapporte Elle de la deuxieme phrase? 

0 
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Appendix 4 

Questionnaire for Students' Ratings 

University of Michigan 
Department of Romance Languages 



Course: French Date: 



Do NOT put down your name or your section number. 

You are asked to complete this questionnaire in order to help your 
instructor to evaluate his teaching skills and the course objectives. 
Please make this evaluation in the same careful and thoughtful manner 
which you would expect from your instructor in his evaluation of your 
work. 

1. Read carefully the description of each item as well as the three 
categories before putting down your mark on the line. 

2. Take each item separately, i.e., disregarding other items or your 
total impression of the instructor or the course. 

3. Place a check mark anywhere on the line for each item. The place 
of your mark should represent your evaluation of the instructor 
or the course. 

4. Generally speaking, the middle category represents the average 
instructor in the department. A check mark to the right of the 
middle would indicate that your instructor is above average in 
that item; conversely, a check mark to the lef t would show that 
in your opinion he is below average. 

5. To help you in evaluating the instructor, you might compare him 
or her to the instructors of other courses keeping in mind certain 
differences resulting from a different subject matter. 

Part One: Instructor 

1. Organization of Class Meetings 



exceptionally 
well organized 

2> Instructor's Interest in the Subject 



noticeable lack satisfactory 

of organization organization 



interest 
seemed mild 



strongly 

interested 



intensely 

interested 
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3. Knowledge of the Subject Matter 



adequate for 
routine class* 
room 


broad and 
thorough knowledge 


knows everything 
there is to know 


4. Does the Instructor 


Speak French in a Way That 


You Can Understand? 


instructor is 
regularly too 
advanced for me 


is occasionally 
too advanced for me 


I can always 
follow readily 


5. Clearness of Explanation 




explanations 
usually not 
clear to me 


meaning usually 
clear to me 


meaning always 
clear to me: ex- 
planation complete 


6. How Interesting Are 


the Class Meetings? 




usually dull 


mildly 

Interesting 


high level of 
Interest maintained 


7. Freshness of Presentation: The Instructor 




follows a 
stale routine 


attempts to bring 
freshness in 
presentation 


uses a wide variety 
of techniques 


8. Self-Confidence 






lack of confidence 
sometimes disturbs 
students 


good self- 
confidence 


admirable self- 
confidence 


9. Tolerance 






unconcerned about 
student opinion 


respects student 
thought 


encourages 
students to ex- 



press thoughts 
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10. Is the Instructor Easy to Talk 



to and Get Help from? 



sometimes difficult 
to get help 



available for and 
helpful in con- 
ferences 



11. Feeling between Instructor and Class 



friendly and 
especially eager 
to be helpful 



teacher and class 
indifferent to 
each other 



teacher and class 
friendly to each 
other 



strong atmosphere 
of mutual good 
will 



12. Hew Well Did You Get to Know Your Instructor, and Vice Versa? 



O 

ERIC 
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very little 


about the same as 
in all other courses 


far better than 
in all other 
courses 


13. Classroom Discussion 




usually a waste 
of time 


often of 
some value 


usually highly 
valuable 


14. Promptness in Returning Homework, Quizzes 


, etc* 


never prompt 


usually prompt 


always prompt 


15. Weekly Quizzes 






poor sampling of 
student's mastery 
of the material 


good 

sampling 


very good 
sampling 


16. Use of Quizzes 


as a Learning Device 




students left 
uncertain of 
mistakes 


usually explains 
and helps students 
improve 


carefully goes over 
exams and helps 
students improve 


17. A General Rating of the Instructor. (Consider all the instructors 
you have had at the University and judge your instructor in comparison to 


poor fair 


good better 


One of the best 
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Appendix 5 

Observer Tally Sheet 



Observation Tally Sheet No. 2_ 



Observer Date Period Teacher 



School Grade No. of Students 



1. Breakdown of Activities 



2. Type of Class 



3. Teachers Performance 

Students participation 
Pace of class 

Quality of grammatical explanation 

4. Teachers Command of French 

5. Teachers Personality 

Imagination 

Humor 

Idiosyncratic Habits 

6 . General Ambiance 




7 



Extra Comments 



H 



o 






03 


















M 




P- 




M 


H* 






H* 


Is) 


• 


o 


Os 


Os 


O 


H 


P- 


00 


00 


M 


Vj 




00 


P- 


P- 


H* 


03 


M 


00 


Cn 


O 


Os 


g 


u> 






Cn 
















Vl 


P- 




H* 


P- 








03 


IS) 

a. 


o 


Is) 


-0 


O 


IS) 


03 


Os 


Cn 


00 


H» 


CT 


VO 

VO 


O 


o 


N) 

Cs 


VO 


IS) 


4> 


P- 


00 


P- 




P- 


M 




M 


OS 






M 


ls) 


03 

• 




€0 


Cn 


O 


IS) 


ro 


vo 


00 


IS) 


VO 


o 


o 

Is) 




Os 


M 

00 


03 


M 


M 


03 


M 


IS) 


a 




oo 


Os 






Is) 










.t* 




Cn 


00 




VJ 


N> 


Is) 


M 


M 


M 


• 


H* 


00 


00 


o 


vl 


M 


H* 


00 


VO 


VJ 


03 


O 


Os 


vi 


Os 


Is) 


Is) 


OS 


oo 


o 


Is) 


V3 








o 












M 


















Cn 




CO 


03 




Is) 


P- 






H* 


IS) 


• 


O 


H 


03 


o 


O 


03 


00 


VJ 


P* 


Is) 


Ci 


VO 

o 


O) 


M 


03 

VO 


O 


M 


00 


O) 


P- 


Cn 


> 






h-> 






l-> 










Os 




P- 


o 




H * 


VO 






H* 


M 4 


• 


o 


VO 


00 


o 


Cn 


03 


00 


VJ 


H* 


VJ 




00 

p- 


o> 


VO 


Cn 

Os 


Cn 


03 


VJ 


Cn 


Os 


P- 


ttf 




CO 


















vl 




Cn 


o 




03 


IS) 


M 


M 




H» 


» 


H* 


Cn 


H* 


o 


o 


Cn 


Os 


Os 


VO 


VO 


8 


H* 


N) 


P- 


p- 


Cn 


H* 


P* 


H» 


03 


o 


P- 






p* 
















U3 


M 




H* 


Cn 


H* 


M 


H* 


ls> 


00 

• 


o 


VO 


*sl 


o 


H» 


P- 


IS) 


M 


P- 


o 


> 


vl 


03 


Os 


N) 


H* 


Cn 


VI 


M 


P- 


Cn 


6 


p- 






VO 














• 


00 


p- 




H* 


00 


H* 


M 




M 


VO 

• 


H* 


•e'- 


N) 


o 


03 


OS 


P- 


O 


Vj 


VO 


►xj 


CO 


en 


Os 


03 


Cn 


H* 


O 


00 


03 


03 




H 1 






H* 














so 


H* 

O 




IS) 


00 


M 


M 


M 


M 


OX 


M 


4> 


K> 


O 


H* 


p- 


00 


Cn 


P- 


Os 


V 


s 


03 


03 


Os 

VI 


00 


H* 


IS) 


VO 


IO 


vl 


S 



n 

■8 



C, 

&> 

JS 



O'* 

fli 



Ratio Findings Based on 0S1 Categories 



er|c 



hJ 




ON 


on 


OJ 




fr 


fr 


fr 


vj 


M 


ho 


VI 


OJ 


OJ 




on 


on 


Vi 


VO 


o 


M 


y-> 


fr 


M 


ON 


on 



M 




fr 


Ov 


fr 




O 


fr 


on 


OJ 


M 


00 


fr 


O 


ISJ 


M 


on 


M 


OJ 


ON 


VO 


o 


ON 


Vi 


OJ 


VI 


VO 



to w ^ 



fr M ls> 



O 



M M O 

VO si QO 



ON 

fr 



N) VO sj O 
00 OJ fr fr 



• 


• 


OJ 

• 


fr* 

• 


OJ 

• 


• 


fr 

OJ 

• 


oo 

hJ 

• 


ON 

Vi 

• 


VO 


hJ 


o 


M 


hJ 


OJ 


ISJ 


hJ 


o 


O 


On 


M 


hJ 


M 


O 


o 


OJ 


on 



M 00 



O M 
ON Ov 



vl fr 
hJ 00 



SO K) 

vj oj 



IsJ 



ON OJ 
00 M 



vi 

on 



N> U) is 



M M M 


On 


hJ H OJ OJ 


• 


0^ OJ On 00 O 




ON 00 hJ O 00 


JA 


M M 


ON 


OJ fr VO M 


• 


hj is> fr on fr 




Vi OV o OJ O 


W 



OJ VO 
H» ON 



iso 

ON 



Vi 

Vi 



Iso N) si 

hJ OJ fr 



ON 

VO 



ON 

OJ 



hJ 

• 


m 


M 

• 


00 

• 


OJ 

• 


• 


M 

• 


ISJ 

• 


H* 

0 


o 


hJ 


ISJ 


VI 


o 


hJ 


ON 


H* 


VO 


vi 


00 


on 


VI 


vl 


vo 


on 


O 


00 



ls> 




M 


VO 


N> 




VO 


fr 


on 


OJ 


H-* 


M 


OJ 


fr 


OJ 


fr 


on 


on 


ls> 


VO 


fr 


Ov 


00 


Ov 


VO 


00 


O 









Ov 












hJ 




M 


00 


OJ 




vi 


O 


o 


ON 


IsJ 


fr 


H* 




OJ 


00 


vo 


o 


VO 


VO 


M 


on 




H 


OJ 


VO 


OJ 



hJ 



OJ 



o 

o 



t/> 

to 



B 

00 

£ 

VO 

a 



t 



rt 

H* 

O 

H* 

P 

CL 

H* 

eg 

CD 

w 

0) H 

CO 0) 

m o' 

04 o 

5 IsJ 

o 

CO 

hJ 

o 

0 ) 

rt 

n> 

00 

o 

H 

H» 

ft) 

CD 



72 



Appendix 6 - continued 



ERIC 

eo 











hh © a 
owe 
n w i 












fil o* 












on h* fD 
O ^ h» 












ft O 












ft O hh 




tn 


-P- 


to 


NS H 


O 

ft Q) O 
© O © 


H 










MOM 












O H» 












ft c w 
OJ S3 ^ 












H ft 3 












H* O 

fl> 1 












W 








NS 






• 


« 


■ 


■ 


■ 






vO 


tn 


NS 


00 


NS 


s 


■P* 


O 


ON 


*sl. 


-P- 


% 




NO 




















NS 






• 








O 


■P- 


vO 


ON 


NS 




o 


tO 

NO 


■P- 


VO 


tn 


CT 












to 


• 


• 


■ 


• 




• 


00 


tn 


VO 


o 


NS 




H 


O 


00 


NS 


tn 


O 




ON 








o 






NS 






4N 


• 


• 


• 


• 




• 




•si 


\o 


vO 


NS 




NS 


*•4 


00 


ON 


NS 


CO 




NS 








CO 






to 


H* 




tn 


• 


• 


• 


• 




• 


NS 


to 


00 


O 


H 




00 


NS 


00 


o 


us 






tn 








> ; 






H 


H 




ON 


■ 


m 


■ 


• 




• 


*s| 


■P* 


00 


H 


NS 




NS 


tn 


ON 


NS 


VO 


►0 


00 








to 


H 




NS 






Vj 




• 


• 


• 




• 


o 


vO 


vO 


00 


H 


S 


■P- 


NS 

H 


VO 


H 


VO 


H 




H 


NS 




00 


• 


• 


■ 


* 




• 


H 


00 


vj 


NS 


to 




■P- 


H 


4* 


VO 


NS 






ON 








b 


H 




H 






VO 


• 


• 


• 


• 




• 


to 


ON 


H 


00 


NS 




H 


H 


NS 


■p* 


■P- 


3 




H 










H 


H» 


NS 


to 




H 


• 


• 


• 


• 




O 


VO 


H 


H 


vj 


NS 


• 


H» 


VO 


00 


to 


ON 






to 








s 



> 


n 




0) 






fl> 


n> 


3 


M 


PU 


M 


H* 




X 


«• 


ON 


t-H 




0) 


1 


3 


o 


<3 


o 


M 


S3 


M 


ft 


0) 



i 



Ratio Findings Based on 0S1 Categories, One-Step Frequencies 



